-- Logs begin at Thu 2022-10-13 03:42:03 UTC, end at Thu 2022-10-13 04:42:14 UTC. -- Oct 13 04:12:34.482390 dv-cinder-860997-1-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 13 04:12:34.491105 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 13 04:12:34 2022] *** Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: nodename: dv-cinder-860997-1-iscsi Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: machine: x86_64 Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: clock source: unix Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: pcre jit disabled Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: detected number of CPU cores: 4 Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: current working directory: / Oct 13 04:12:34.493076 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: detected binary path: /usr/bin/uwsgi-core Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: your processes number limit is 47680 Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: your memory page size is 4096 bytes Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: detected max file descriptor number: 2048 Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: lock engine: pthread robust mutexes Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: thunder lock: enabled Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 13 04:12:34.493741 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 13 04:12:34.532641 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: Python main interpreter initialized at 0x56251ddbc9c0 Oct 13 04:12:34.532641 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: python threads support enabled Oct 13 04:12:34.532641 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: your server socket listen backlog is limited to 100 connections Oct 13 04:12:34.532641 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: your mercy for graceful operations on workers is 90 seconds Oct 13 04:12:34.533047 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: mapped 403077 bytes (393 KB) for 2 cores Oct 13 04:12:34.533047 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: *** Operational MODE: preforking *** Oct 13 04:12:34.533292 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: *** uWSGI is running in multiple interpreter mode *** Oct 13 04:12:34.533292 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: spawned uWSGI master process (pid: 103682) Oct 13 04:12:34.533472 dv-cinder-860997-1-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 13 04:12:34.535211 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: spawned uWSGI worker 1 (pid: 103683, cores: 1) Oct 13 04:12:34.535211 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: spawned uWSGI worker 2 (pid: 103684, cores: 1) Oct 13 04:12:34.535211 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 13 04:12:35.196455 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 13 04:12:35.213324 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 13 04:12:36.283146 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:12:36.283146 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: from cryptography.utils import int_from_bytes Oct 13 04:12:36.283760 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:12:36.283760 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: from cryptography.utils import int_from_bytes Oct 13 04:12:36.302805 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:12:36.302805 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: from cryptography.utils import int_from_bytes Oct 13 04:12:36.303275 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:12:36.303275 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: from cryptography.utils import int_from_bytes Oct 13 04:12:38.796307 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 13 04:12:38.796946 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 13 04:12:38.797081 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 13 04:12:38.797438 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 13 04:12:38.797555 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 13 04:12:38.797934 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 13 04:12:38.798021 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 13 04:12:38.798114 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.798285 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 13 04:12:38.798377 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.798654 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 13 04:12:38.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.798914 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 13 04:12:38.799025 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.799304 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.799399 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.799694 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.799787 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.800032 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.800141 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.800348 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.800576 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.800677 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.800913 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801001 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801219 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801305 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801539 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860997-1-iscsi {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801627 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801855 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.801949 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.802160 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.802276 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.802456 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.802554 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.802776 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.802870 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.803076 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.803171 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860997-1-iscsi {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.803393 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.803482 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.803691 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.803779 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.804005 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.804181 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.804323 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805304 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805411 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805557 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805621 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805691 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805766 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.805916 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806089 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806181 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806340 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806433 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806649 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806739 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.806996 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.807184 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.807380 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.807533 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.807711 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.807888 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860997-1-iscsi {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.808100 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.808302 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.808453 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.808631 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.808807 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.808982 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.809203 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.809394 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.809589 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860997-1-iscsi {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.809783 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.809973 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.810126 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.810312 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.810495 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.810623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.810752 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.810752 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811016 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811149 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811279 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811402 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811685 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811814 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.811937 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812072 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812201 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812330 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812454 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812585 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812717 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.812840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813004 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813128 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813264 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813379 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813508 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813636 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813759 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.813881 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814001 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814131 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814287 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814413 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814542 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814713 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.814841 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.815020 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.815148 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.815821 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.815949 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.816126 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.816251 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.816443 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.816572 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.816747 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.816880 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817050 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817177 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817353 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817480 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817652 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817785 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.817950 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.818084 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.818273 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.818401 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.818575 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.818702 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.818879 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819015 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819191 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819315 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819499 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819637 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819856 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.819981 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.820105 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.820226 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.820386 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.820516 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.820693 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.820830 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821008 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821134 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821308 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821440 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821629 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821760 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.821937 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822070 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822251 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822377 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822559 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822687 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822873 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.822999 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.823176 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.823315 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.823476 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.823602 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.823776 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.823902 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824089 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824223 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824414 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824548 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824712 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824845 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.824969 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825103 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825226 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825349 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825521 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825649 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825833 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.825969 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.826133 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.826375 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.826504 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.826633 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.826756 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.826886 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827010 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827131 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827310 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827453 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827611 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827739 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.827910 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828037 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828209 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828343 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828508 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828636 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828812 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.828945 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.829110 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.829236 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.829410 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.829538 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.829710 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.829835 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.830007 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.830141 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.830334 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.830462 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.830643 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.830769 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.831523 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.831685 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.831811 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.831955 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.832166 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.832301 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.832465 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.832599 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.832780 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.832908 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833080 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833206 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833379 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833514 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833691 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833824 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.833988 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.834114 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.834318 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.834459 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.834631 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.834759 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.834934 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835071 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835234 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835373 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835546 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835673 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835840 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.835975 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.836140 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.836278 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.836448 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.836581 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.836747 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.836875 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837047 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837183 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837341 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837467 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837638 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837765 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.837935 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838071 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838264 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838408 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838534 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838656 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838838 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.838967 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.839132 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.839260 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.839443 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.839568 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.839748 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.839876 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840047 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840172 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840347 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840471 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840641 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.840936 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.841063 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.841254 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.841380 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.841576 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.841710 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.841886 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.842019 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.842192 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.842355 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.842521 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.842647 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.842839 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:12:38.842965 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.843154 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.843292 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.843464 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.843590 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.843774 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.843902 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.844095 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.844236 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.844403 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.844528 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.844708 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.844842 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845014 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845140 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845327 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845462 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845640 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845766 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.845966 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.846092 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.846299 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.846522 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.846657 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.846788 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.846957 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.847096 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.847265 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.847392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.847578 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.847712 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.847899 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848019 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848215 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848341 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848522 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848648 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848863 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.848997 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.849201 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.849328 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.849508 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.849643 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.849827 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.849961 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.850143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.850325 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.850460 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.850585 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.850767 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.850895 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.851073 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.851210 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.851406 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.851543 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.851713 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.851840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.852020 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.852150 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.852335 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.852718 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.852894 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853030 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853203 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853328 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853508 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853647 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853813 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.853947 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.854116 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.854265 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.854447 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.854592 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.854755 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.854889 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.855085 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.855210 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.855391 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.855516 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.855705 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.855830 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856010 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856135 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856316 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856443 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856624 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856757 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.856927 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857052 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857233 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857361 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857537 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857663 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857837 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.857964 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.858136 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.858310 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.858458 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.858585 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.858770 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.858898 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859071 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859206 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859385 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859512 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859688 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859823 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.859995 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.860122 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.860301 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.860427 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.860611 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.860742 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.860911 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861003 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861215 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861302 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861521 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861621 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861825 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.861913 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.862182 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.862458 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.862622 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.862791 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.862926 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863071 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863202 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863348 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863500 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863644 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863780 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.863930 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.864140 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.864290 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.864448 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.864573 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.864695 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.864909 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865047 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865173 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865308 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865432 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865561 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865684 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865842 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.865977 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.866150 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.866308 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.866497 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.866625 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.866805 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.866941 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.867110 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.867246 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.867422 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.867540 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.867741 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.867876 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868041 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868173 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868361 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868487 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868674 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868811 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.868986 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.869111 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.869288 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.869423 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.869593 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.869800 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.869925 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870047 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870170 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870320 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870532 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870659 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870779 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.870900 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871073 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871198 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871373 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871509 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871673 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871800 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.871974 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.872101 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.872276 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.872402 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.872594 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.872719 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.872900 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873033 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873198 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873331 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873498 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873624 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873800 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.873933 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.874107 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.874249 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.874445 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.874570 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.874752 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.874889 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875059 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875185 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875364 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875496 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875675 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875811 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.875986 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.876111 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.876297 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.876430 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.876602 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.876738 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.876906 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.877033 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.877211 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.877337 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.877515 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.877635 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.878378 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.878507 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.878692 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.878870 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879010 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879215 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879341 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879532 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879658 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879848 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.879985 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.880162 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.880289 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.880483 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.880609 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.880790 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.880925 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.881111 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.881237 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.881425 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.881551 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.881730 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.881870 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882046 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882173 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882373 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882499 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882675 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882803 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.882981 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.883110 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.883287 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.883413 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.883591 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.883718 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.883896 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884024 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884334 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884514 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884641 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884827 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.884954 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.885130 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.885257 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.885441 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.885566 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.885743 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.885867 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.886059 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.886194 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.886382 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.886507 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.886691 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.886818 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887007 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887144 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887319 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887451 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887624 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.887937 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888076 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888240 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888367 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888543 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888676 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888847 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.888981 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.889154 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.889290 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.889455 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.889581 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.889757 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.889884 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.890060 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.890187 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.890394 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.890522 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.890700 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.890828 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891005 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891131 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891306 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891434 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891615 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.891920 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892046 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892217 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892349 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892516 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892650 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892819 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.892945 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.893127 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.893260 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.893505 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.893635 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.893829 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.893956 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.894142 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.894406 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.894533 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.894655 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.894778 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.894908 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.895068 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.895208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.895392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.895519 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.895703 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.895829 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896034 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896168 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896329 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896455 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896652 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896785 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.896960 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897095 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897239 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897369 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897553 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897693 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897871 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.897997 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.898175 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.898318 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.898509 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.898635 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.898843 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.898969 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.899114 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.899241 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.899432 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.899566 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.899742 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.899877 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900046 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900173 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900351 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900477 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900657 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900785 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.900964 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.901090 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.901271 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.901404 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.901584 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.901710 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.901888 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902013 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902192 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902354 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902521 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902648 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902820 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.902947 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.903126 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.903261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.903434 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.903552 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.903757 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.903997 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.904125 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.904326 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.904454 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.904638 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.904766 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.904952 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.905087 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.905264 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.905390 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.905581 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.905717 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.905903 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.906152 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.906318 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.906508 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.906635 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.906824 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.906950 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.907127 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.907262 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.907431 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.907558 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.907731 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.907871 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.908036 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.908164 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.908336 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.908464 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.908638 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.908765 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.909436 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.909570 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.909814 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.910072 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.910199 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.910445 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.910571 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.910758 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.910901 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911065 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911193 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911373 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911500 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911678 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911823 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.911991 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.912117 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.912296 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.912423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.912602 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.912737 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.912905 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913040 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913211 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913337 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913526 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913646 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913832 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.913959 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.914144 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.914301 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.914476 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.914611 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.914800 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.914927 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.915113 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.915240 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.915420 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.915548 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.915725 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.915863 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.916031 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.916165 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.916380 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.916508 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.916688 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.916814 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917017 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917154 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917326 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917452 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917632 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917758 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.917944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.918080 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.918278 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.918407 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.918584 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.918711 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.918892 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919019 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919202 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919330 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919507 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919635 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919812 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.919938 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.920114 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.920240 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.920415 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.920541 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.920715 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.920842 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921019 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921144 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921324 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921448 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921629 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921757 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.921926 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922051 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922254 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922396 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922564 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922691 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922862 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.922995 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.923169 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.923375 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.923501 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.923623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.923743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.923865 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.924055 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.924192 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.924354 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.924480 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.925209 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.925341 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.925525 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.925831 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.925958 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.926148 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.926302 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.926481 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.926625 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.926788 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.926922 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.927094 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.927220 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.927398 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.927538 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.927704 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.927831 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928006 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928123 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928305 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928438 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928609 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928736 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.928914 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929038 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929218 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929361 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929527 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929661 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929827 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.929951 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.930126 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.930281 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.930450 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.930582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.930754 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.930881 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931056 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931183 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931358 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931485 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931656 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931782 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.931957 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932083 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932255 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932381 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932555 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932680 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932853 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.932979 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.933151 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.933276 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.933453 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.933588 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.933750 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.933868 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934053 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934185 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934358 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934493 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934690 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934815 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.934992 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.935116 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.935294 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.935428 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.935601 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.935734 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.935905 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936032 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936232 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936366 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936540 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936666 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936852 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.936972 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.937164 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.937265 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.937467 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.937572 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.937780 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.937877 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.938089 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.938225 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.938424 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.938521 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.938720 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.938806 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939021 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939419 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939644 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939739 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.939948 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.940034 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.940274 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.940366 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.940581 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.940862 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.940957 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.941201 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.941295 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.941511 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.941597 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.941832 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.941919 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.942139 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.942246 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.942473 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.942560 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.942782 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.942877 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.943089 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.943182 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.943395 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.943480 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.943703 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.943789 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944008 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944094 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944316 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944408 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944622 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944708 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.944925 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945011 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945234 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945437 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945528 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945608 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945831 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.945923 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.946134 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.946247 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.946477 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.946564 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.946779 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.946867 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947086 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947180 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947387 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947472 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947694 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947788 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.947996 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.948099 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.948304 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.948432 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.948613 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.948717 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.948924 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.949029 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.949231 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.949325 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.949610 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.949695 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.949915 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950010 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950248 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950338 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950563 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950649 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950862 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.950949 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.951171 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.951276 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.951486 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.951571 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.951801 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.951888 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.952104 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.952192 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.952456 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.952615 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.952759 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.952845 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953065 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953152 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953387 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953481 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953688 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953774 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.953988 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.954070 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.954314 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.954428 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.954635 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.954722 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.954938 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.955023 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.955245 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.955341 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.955537 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.955655 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.955884 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.956079 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.956198 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.956524 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.956651 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.956940 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.957118 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.957252 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.957431 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.957559 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.957744 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.957974 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.958059 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.958313 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.958441 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.958626 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.958766 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959049 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959230 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959359 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959557 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959675 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959886 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.959995 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.960195 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.960316 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.960514 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.960623 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.960826 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.960934 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.961152 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.961240 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.961479 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.961574 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.961792 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.961878 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.962110 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.962196 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.962444 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.962541 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.962771 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.962866 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.963097 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.963183 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.963411 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.963577 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.963736 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.963832 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964078 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964209 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964382 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964518 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964683 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964783 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.964989 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.965076 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.965289 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.965385 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.965597 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.965703 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.965917 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966004 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966238 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966330 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966543 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966632 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966853 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.966941 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.967158 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.967254 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.967459 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.967546 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.967759 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.967847 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968080 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968167 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968378 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968465 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968681 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968774 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.968987 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.969077 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.969302 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.969387 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.969603 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.969698 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.969914 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970000 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970250 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970346 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970561 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970659 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970864 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.970953 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.971168 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.971262 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.971984 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.972148 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.972320 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.972536 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.972624 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.972919 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973018 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973226 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973313 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973573 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973659 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973879 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.973974 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.974247 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.974340 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.974554 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.974641 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.974887 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.974986 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.975198 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.975290 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.975507 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.975606 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.975825 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.975934 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.976147 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.976235 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.976450 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.976537 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.976754 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.976842 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.977061 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.977148 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.977609 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.977702 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.977921 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978008 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978235 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978325 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978545 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978633 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978845 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.978934 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.979148 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.979236 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.979447 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.979534 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.979748 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.979842 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980054 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980142 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980359 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980455 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980662 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980748 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.980963 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981057 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981266 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981361 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981566 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981653 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981866 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.981951 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.982174 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.982290 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.982500 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.982595 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.982823 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.982916 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.983137 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.983232 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.983452 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.983538 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.983761 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.983849 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.984069 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.984163 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.984382 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.984469 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.984694 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.984782 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985005 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985094 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985316 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985403 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985623 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985755 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.985939 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.986060 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.986269 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.986366 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.986586 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.986689 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.986902 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.987692 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.987812 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.988057 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.988289 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.988582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.988675 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.988917 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989009 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989236 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989329 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989551 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989653 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989869 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.989963 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.990185 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.990308 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.990518 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.990610 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.990829 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.990920 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.991142 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.991242 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.991455 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.991547 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.991768 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.991869 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.992090 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.992190 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.992412 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.992503 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.992733 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.992823 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993050 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993152 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993364 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993456 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993677 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.993996 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.994085 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.994352 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.994464 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.994658 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.994764 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.994972 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.995073 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.995285 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.995376 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.995595 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.995696 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.995902 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996004 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996211 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996304 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996521 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996627 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996832 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.996932 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.997136 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.997228 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.997444 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.997545 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.997752 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.997850 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.998058 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.998149 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.998395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.998560 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.998707 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.998850 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999023 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999175 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999329 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999481 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999630 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999774 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:38.999917 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.000074 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.000271 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.000423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.000579 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.000723 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.001047 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.001206 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.001375 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.001528 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.001700 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.001858 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.002004 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.002157 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.002369 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.002519 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.002663 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.002924 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.003078 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.003232 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.003378 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.003593 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.003746 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.003891 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004034 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004187 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004333 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004484 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004638 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004802 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.004957 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005103 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005247 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005390 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005534 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005839 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.005984 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.006128 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.006299 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.006563 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.006825 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.006999 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.007174 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.007328 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.007480 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.007626 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.007806 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.007971 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.008117 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.008259 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.008403 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.008547 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.008722 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.008876 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009032 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009177 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009320 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009482 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009639 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009786 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.009940 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.010085 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.010278 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.010431 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.010585 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.010730 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.010886 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011042 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011187 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011330 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011483 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011628 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011771 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.011915 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012065 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012222 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012367 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012510 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012654 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012808 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.012960 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.013118 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.013263 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.013408 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.013553 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.013703 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.013849 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014001 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014147 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014324 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014470 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014615 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014780 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.014936 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015090 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015236 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015380 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015525 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015677 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015822 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.015975 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.016121 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.016265 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.016416 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.016569 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.016724 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.016869 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017020 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017165 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017309 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017462 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017607 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017751 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.017895 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.018039 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.018183 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.018419 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.018568 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.018724 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.018870 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.019015 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.019205 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.019453 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.019606 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.019751 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.019907 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.020054 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.020208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.020400 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.020553 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.020712 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.020928 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.021241 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.021403 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.021549 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.021692 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.021881 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.022033 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.022287 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.022458 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.022605 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.022796 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.022945 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.023101 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.023320 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.023467 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.023611 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.023763 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.023909 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024053 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024205 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024351 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024495 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024640 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024784 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.024942 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.025101 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.025246 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.025391 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.025534 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.025679 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.025835 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026007 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026161 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026332 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026478 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026634 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026790 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.026949 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027094 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027237 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027380 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027523 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027677 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027822 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.027966 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.028111 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.028255 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.028407 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.028588 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.028749 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.028894 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029039 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029190 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029344 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029489 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029633 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029778 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.029922 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.030075 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.030290 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.030443 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.030588 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.030743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.030890 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031034 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031186 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031332 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031483 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031629 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031773 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.031926 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.032073 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.032217 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.032361 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.032514 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.032694 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.032917 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.033071 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.033224 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.033376 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.033544 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.033708 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.033854 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.034541 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.034743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.035145 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.035461 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.035614 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.035788 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.035947 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.036103 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.036248 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.036404 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.036556 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.036739 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.036891 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.037068 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.037220 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.037383 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.037566 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.037714 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.037859 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038002 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038155 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038350 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038512 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038659 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038811 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.038956 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039109 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039266 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039410 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039554 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039700 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039844 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.039996 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.040142 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.040297 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.040449 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.040595 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.040770 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.040921 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041065 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041219 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041364 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041509 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041664 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041810 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.041954 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.042097 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.042293 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.042455 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.042609 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.042754 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.042898 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043041 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043202 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043348 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043501 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043654 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043799 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.043943 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044087 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044240 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044392 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044538 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044682 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044825 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.044968 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.045130 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.045275 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.045427 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.045573 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.045717 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.045860 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046013 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046158 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046435 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046542 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046630 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046713 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046806 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046889 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.046968 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047048 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047231 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047336 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047547 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047636 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047872 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.047960 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.048195 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.048292 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.048513 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.048601 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.048825 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.048912 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.049130 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.049224 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.049438 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.049526 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.050439 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.050533 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.050860 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051018 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051114 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051237 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051383 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051565 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051697 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.051894 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052028 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052202 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052358 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052525 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052685 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052840 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.052981 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.053156 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.053301 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.053471 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.053624 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.053780 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.053942 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.054106 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.054282 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.054505 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.054618 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.054816 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.054931 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.055126 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.055246 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.055434 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.055558 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.055743 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.055871 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056056 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056200 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056365 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056501 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056674 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056819 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.056989 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.057132 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.057303 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.057450 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.057629 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.057769 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.057954 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.058100 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.058289 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.058458 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.058614 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.058772 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.058941 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.059103 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.059289 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.059469 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.059619 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.059766 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.059921 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.060074 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.060246 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.060430 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.060577 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.060722 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.060916 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.061096 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.061244 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.061397 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.061542 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 13 04:12:39.061687 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:12:39.061838 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 13 04:12:39.214179 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO dbcounter [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Registered counter for database nova_api Oct 13 04:12:39.215768 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO dbcounter [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Registered counter for database nova_api Oct 13 04:12:39.234675 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_db.sqlalchemy.engines [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103683) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.236354 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_db.sqlalchemy.engines [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103684) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.241880 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writer thread running {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.243737 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writer thread running {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.298249 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.context [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),1311a648-eb92-4594-8c2f-3c55fab1c340(cell1) {{(pid=103683) load_cells /opt/stack/nova/nova/context.py:464}} Oct 13 04:12:39.300830 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.context [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),1311a648-eb92-4594-8c2f-3c55fab1c340(cell1) {{(pid=103684) load_cells /opt/stack/nova/nova/context.py:464}} Oct 13 04:12:39.302621 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_concurrency.lockutils [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103683) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:12:39.303098 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_concurrency.lockutils [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103683) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:12:39.304654 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_concurrency.lockutils [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103683) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:12:39.304753 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:12:39.305162 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:12:39.305386 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_concurrency.lockutils [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103683) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:12:39.305835 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_concurrency.lockutils [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103683) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:12:39.306724 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:12:39.307453 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:12:39.307841 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_concurrency.lockutils [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103683) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:12:39.307942 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:12:39.309820 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO dbcounter [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Registered counter for database nova_cell0 Oct 13 04:12:39.309922 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:12:39.311810 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO dbcounter [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Registered counter for database nova_cell0 Oct 13 04:12:39.313861 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO dbcounter [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Registered counter for database nova_cell1 Oct 13 04:12:39.316749 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO dbcounter [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Registered counter for database nova_cell1 Oct 13 04:12:39.321663 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_db.sqlalchemy.engines [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103683) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.323446 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_db.sqlalchemy.engines [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103684) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.325210 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_db.sqlalchemy.engines [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103683) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.328093 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writer thread running {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.328210 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writer thread running {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.328940 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_db.sqlalchemy.engines [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103684) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.466512 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writer thread running {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.471007 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writer thread running {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.478225 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO dbcounter [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Registered counter for database nova_cell0 Oct 13 04:12:39.481166 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO dbcounter [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Registered counter for database nova_cell0 Oct 13 04:12:39.486991 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG oslo_db.sqlalchemy.engines [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103683) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.489728 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_db.sqlalchemy.engines [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103684) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:12:39.493854 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writer thread running {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.494709 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writer thread running {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:12:39.841551 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: WARNING keystonemiddleware.auth_token [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:12:39.845265 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: WARNING keystonemiddleware.auth_token [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:12:39.849734 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: WARNING keystonemiddleware.auth_token [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:12:39.853333 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: WARNING keystonemiddleware.auth_token [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:12:40.196530 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: WARNING keystonemiddleware.auth_token [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:12:40.201726 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: WARNING keystonemiddleware.auth_token [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:12:40.202475 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: WARNING keystonemiddleware.auth_token [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:12:40.207944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56251ddbc9c0 pid: 103684 (default app) Oct 13 04:12:40.208312 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: WARNING keystonemiddleware.auth_token [None req-8ab049c0-7572-4c00-b85f-e036d79b841f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:12:40.210029 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:12:40.211249 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] 10.222.0.27 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001787 Oct 13 04:12:40.211662 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 788 bytes} [Thu Oct 13 04:12:40 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:12:40.213743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56251ddbc9c0 pid: 103683 (default app) Oct 13 04:14:01.791567 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:01.792638 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Oct 13 04:14:01.793039 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 2/2] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:01.810767 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_api:SELECT=3 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:01.811591 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_cell1:SELECT=3 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:01.812905 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_cell0:SELECT=3 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:01.813936 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:02.719703 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-9642440f-0f08-4def-9e2b-78395cd739dd admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:02.753315 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-9642440f-0f08-4def-9e2b-78395cd739dd admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.952557 Oct 13 04:14:02.754083 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 1/3] 10.222.0.27 () {60 vars in 1292 bytes} [Thu Oct 13 04:14:01 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 954 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:05.391785 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:05.393207 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-caee3042-d7c7-4e87-958a-1be04e8a631e None None] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001799 Oct 13 04:14:05.393880 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 3/4] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:05 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:05.843415 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-29c34ea2-baa0-4e5c-830d-988fd97206b6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "os-flavor-access:is_public": true, "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "swap": 0, "vcpus": 1, "disk": 1, "rxtx_factor": 1.0}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:05.892297 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-29c34ea2-baa0-4e5c-830d-988fd97206b6 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.492088 Oct 13 04:14:05.892870 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 2/5] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:05 2022] POST /compute/v2.1/flavors => generated 407 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 13 04:14:06.122710 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-55f58738-ea9f-459d-b8d9-ded8c8d5ae8c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:06.129153 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_api:SELECT=3 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:06.130164 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_cell0:SELECT=3 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:06.130898 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_cell1:SELECT=3 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:06.131594 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:06.197503 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-55f58738-ea9f-459d-b8d9-ded8c8d5ae8c admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.296271 Oct 13 04:14:06.198098 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 4/6] 10.222.0.27 () {64 vars in 1336 bytes} [Thu Oct 13 04:14:05 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:08.836393 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-29c34ea2-baa0-4e5c-830d-988fd97206b6 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:08.837402 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-29c34ea2-baa0-4e5c-830d-988fd97206b6 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Oct 13 04:14:08.837808 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 3/7] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:09.551435 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-4c2845ac-e97d-46c2-8382-ecb02e270fd9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0, "id": "d1", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512, "disk": 5, "name": "ds512M"}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:09.594173 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-4c2845ac-e97d-46c2-8382-ecb02e270fd9 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.749577 Oct 13 04:14:09.595193 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 5/8] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:08 2022] POST /compute/v2.1/flavors => generated 404 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 13 04:14:09.607630 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-0ffe2746-6b1b-4733-b2e6-e4cd5ffdadf2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:09.652389 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-0ffe2746-6b1b-4733-b2e6-e4cd5ffdadf2 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.048027 Oct 13 04:14:09.662820 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 4/9] 10.222.0.27 () {64 vars in 1336 bytes} [Thu Oct 13 04:14:09 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:12.284506 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-4c2845ac-e97d-46c2-8382-ecb02e270fd9 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:12.285532 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-4c2845ac-e97d-46c2-8382-ecb02e270fd9 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Oct 13 04:14:12.285934 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 6/10] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:12.710007 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-060ca2e2-64c6-4402-b6b0-81e59946139a admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "vcpus": 1, "os-flavor-access:is_public": true, "disk": 10, "id": "d2", "swap": 0, "name": "ds1G", "rxtx_factor": 1.0}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:12.731871 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-060ca2e2-64c6-4402-b6b0-81e59946139a admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.439902 Oct 13 04:14:12.732446 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 5/11] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:12 2022] POST /compute/v2.1/flavors => generated 404 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:14:12.745676 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-136326ec-5672-459e-ab24-4c2f6eab540d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:12.777599 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-136326ec-5672-459e-ab24-4c2f6eab540d admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036082 Oct 13 04:14:12.778199 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 7/12] 10.222.0.27 () {64 vars in 1336 bytes} [Thu Oct 13 04:14:12 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:15.415463 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-060ca2e2-64c6-4402-b6b0-81e59946139a admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:15.417073 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-060ca2e2-64c6-4402-b6b0-81e59946139a admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002053 Oct 13 04:14:15.418451 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 6/13] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:15 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:15.852571 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-58b58e71-b85c-4094-904e-5571deebed4c admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "vcpus": 2, "name": "ds2G", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "rxtx_factor": 1.0, "disk": 10, "os-flavor-access:is_public": true, "swap": 0}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:15.873144 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-58b58e71-b85c-4094-904e-5571deebed4c admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.448591 Oct 13 04:14:15.873741 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 8/14] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:15 2022] POST /compute/v2.1/flavors => generated 404 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:14:15.885766 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1c260eb5-9876-4303-bd72-56c40eb8989d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:15.915153 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1c260eb5-9876-4303-bd72-56c40eb8989d admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032938 Oct 13 04:14:15.915738 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 7/15] 10.222.0.27 () {64 vars in 1336 bytes} [Thu Oct 13 04:14:15 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:18.639217 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-58b58e71-b85c-4094-904e-5571deebed4c admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:18.640954 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-58b58e71-b85c-4094-904e-5571deebed4c admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002342 Oct 13 04:14:18.641429 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 9/16] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:18 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:19.098535 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-eff145c9-0098-4c29-b356-f814655872ae admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "rxtx_factor": 1.0, "swap": 0, "disk": 20, "id": "d4", "ram": 4096, "os-flavor-access:is_public": true, "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:19.117979 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-eff145c9-0098-4c29-b356-f814655872ae admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.469280 Oct 13 04:14:19.118559 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 8/17] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:18 2022] POST /compute/v2.1/flavors => generated 404 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 13 04:14:19.132819 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-2eb87b3f-58c7-4433-9c63-de253d16938b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:19.165819 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-2eb87b3f-58c7-4433-9c63-de253d16938b admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037567 Oct 13 04:14:19.166572 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 10/18] 10.222.0.27 () {64 vars in 1336 bytes} [Thu Oct 13 04:14:19 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:22.022400 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-eff145c9-0098-4c29-b356-f814655872ae admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:22.023690 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-eff145c9-0098-4c29-b356-f814655872ae admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001923 Oct 13 04:14:22.024166 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 9/19] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:22 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:22.507736 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-62e4e805-31db-40c3-b9ff-b24458939a1e admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:22.534809 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-62e4e805-31db-40c3-b9ff-b24458939a1e admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.502121 Oct 13 04:14:22.535390 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 11/20] 10.222.0.27 () {60 vars in 1292 bytes} [Thu Oct 13 04:14:22 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:14:25.212476 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-eff145c9-0098-4c29-b356-f814655872ae admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:25.213443 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-eff145c9-0098-4c29-b356-f814655872ae admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001509 Oct 13 04:14:25.213840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 10/21] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:25.617973 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-305bada7-4549-4595-b64c-9f26f65fccb6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 1, "vcpus": 1, "id": "1", "name": "m1.tiny", "ram": 512, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:25.638167 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-305bada7-4549-4595-b64c-9f26f65fccb6 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.417457 Oct 13 04:14:25.638850 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 12/22] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:25 2022] POST /compute/v2.1/flavors => generated 402 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:14:25.651669 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-132a5c43-49ae-4097-a915-7c844480f0d1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:25.679706 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-132a5c43-49ae-4097-a915-7c844480f0d1 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031772 Oct 13 04:14:25.680281 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 11/23] 10.222.0.27 () {64 vars in 1333 bytes} [Thu Oct 13 04:14:25 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:14:28.292170 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-305bada7-4549-4595-b64c-9f26f65fccb6 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:28.293173 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-305bada7-4549-4595-b64c-9f26f65fccb6 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Oct 13 04:14:28.294033 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:28.680179 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-241fb0cc-1a25-43c6-adb9-1e1056fb7b72 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "2", "disk": 20, "os-flavor-access:is_public": true, "ram": 2048, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "m1.small"}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:28.701132 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-241fb0cc-1a25-43c6-adb9-1e1056fb7b72 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.400586 Oct 13 04:14:28.702358 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 12/25] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:28 2022] POST /compute/v2.1/flavors => generated 405 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:14:28.714133 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-77bfd486-d8ce-432b-8bef-7596eb2a750a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:28.743380 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-77bfd486-d8ce-432b-8bef-7596eb2a750a admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032935 Oct 13 04:14:28.744309 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 14/26] 10.222.0.27 () {64 vars in 1333 bytes} [Thu Oct 13 04:14:28 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:14:31.389751 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-241fb0cc-1a25-43c6-adb9-1e1056fb7b72 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:31.390783 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-241fb0cc-1a25-43c6-adb9-1e1056fb7b72 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001586 Oct 13 04:14:31.391182 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 13/27] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:31.851059 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-74983678-e2f8-4701-95a9-4cebea9442ae admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "3", "swap": 0, "vcpus": 2, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 40, "name": "m1.medium", "ram": 4096}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:31.888654 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-74983678-e2f8-4701-95a9-4cebea9442ae admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.491412 Oct 13 04:14:31.890679 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 15/28] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:31 2022] POST /compute/v2.1/flavors => generated 406 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:14:31.906565 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-4428129f-32e4-499c-869a-c1912d0f0990 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:31.912128 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:31.946404 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-4428129f-32e4-499c-869a-c1912d0f0990 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040087 Oct 13 04:14:31.946404 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 14/29] 10.222.0.27 () {64 vars in 1333 bytes} [Thu Oct 13 04:14:31 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:34.635748 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-74983678-e2f8-4701-95a9-4cebea9442ae admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:34.636772 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-74983678-e2f8-4701-95a9-4cebea9442ae admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Oct 13 04:14:34.637363 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 16/30] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:35.106268 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-6f2c7485-25ec-4e54-a9ca-d1312ad872e0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 8192, "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 80, "id": "4", "swap": 0, "name": "m1.large", "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:35.126467 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-6f2c7485-25ec-4e54-a9ca-d1312ad872e0 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.481199 Oct 13 04:14:35.127318 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 15/31] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:34 2022] POST /compute/v2.1/flavors => generated 405 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 13 04:14:35.140309 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-eb2798e8-ec11-4d54-93b2-d1969ca84997 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:35.165285 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-eb2798e8-ec11-4d54-93b2-d1969ca84997 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029347 Oct 13 04:14:35.165876 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 17/32] 10.222.0.27 () {64 vars in 1333 bytes} [Thu Oct 13 04:14:35 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:37.855515 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-6f2c7485-25ec-4e54-a9ca-d1312ad872e0 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:37.857151 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-6f2c7485-25ec-4e54-a9ca-d1312ad872e0 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002170 Oct 13 04:14:37.857744 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 16/33] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:37 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:38.293678 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-f3998760-9af3-41fa-b828-d73ef61937f2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "swap": 0, "disk": 160, "name": "m1.xlarge", "id": "5", "ram": 16384, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:38.304866 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:38.330288 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-f3998760-9af3-41fa-b828-d73ef61937f2 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.459648 Oct 13 04:14:38.330288 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 18/34] 10.222.0.27 () {64 vars in 1283 bytes} [Thu Oct 13 04:14:37 2022] POST /compute/v2.1/flavors => generated 408 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:14:38.337065 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-e6bcb34d-5dfa-427a-91f8-5fa3dfafff95 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:14:38.365885 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-e6bcb34d-5dfa-427a-91f8-5fa3dfafff95 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032955 Oct 13 04:14:38.366813 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 17/35] 10.222.0.27 () {64 vars in 1333 bytes} [Thu Oct 13 04:14:38 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:14:54.741117 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-f3998760-9af3-41fa-b828-d73ef61937f2 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:54.742139 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-f3998760-9af3-41fa-b828-d73ef61937f2 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001513 Oct 13 04:14:54.742579 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 19/36] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:14:54 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:54.753677 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:14:55.131056 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-28641abc-0ef2-4d9c-a870-510384399986 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:55.149148 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-28641abc-0ef2-4d9c-a870-510384399986 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.400144 Oct 13 04:14:55.149688 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 18/37] 10.222.0.27 () {60 vars in 1292 bytes} [Thu Oct 13 04:14:54 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:14:56.458710 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-f3998760-9af3-41fa-b828-d73ef61937f2 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:56.459775 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-f3998760-9af3-41fa-b828-d73ef61937f2 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001539 Oct 13 04:14:56.460228 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 20/38] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:14:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:56.982281 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-919f09a0-ebc7-410a-86f0-9ed0c04881b9 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:56.987743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.wsgi [None req-919f09a0-ebc7-410a-86f0-9ed0c04881b9 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-919f09a0-ebc7-410a-86f0-9ed0c04881b9 admin admin] Request method failure captured: Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Accept: application/json Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Accept-Encoding: gzip, deflate Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Connection: keep-alive Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Content-Length: 0 Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Host: 10.222.0.27 Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Openstack-Api-Version: compute 2.92 Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Openstack-System-Scope: None Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: User-Agent: python-novaclient Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Auth-Token: gAAAAABjR5DAZ1RLSglenAik29sKRsFKKqTbkGbzwTdOi6Y9qHzoNxzSjDXVLgse08fLPvEvEavCOWnayoJtF5EUFt4h84DDg7wCopX-9DE3dHtkksaE5jsqX3xsUh8hAyf_xfxMtan3Ppz0DgacsXz57MTWw6l5v-3X5ioRxjeNq7_lJNEKO0I Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Domain-Id: None Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Domain-Name: None Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Identity-Status: Confirmed Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Is-Admin-Project: True Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Openstack-Nova-Api-Version: 2.92 Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Domain-Id: default Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Domain-Name: Default Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Name: admin Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Role: member,admin,reader Oct 13 04:14:56.990922 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Roles: member,admin,reader Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8ba823f313144da3b683bacef8fd62e0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant: admin Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant-Name: admin Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User: admin Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Domain-Id: default Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Domain-Name: Default Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Id: 0e2d979563d6421995451a0e86bf3022 Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Name: admin Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: method: > Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: exception: Flavor 42 could not be found. Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: action_args: {'id': '42'} Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:14:56.992530 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:56.993582 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:56.994770 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-919f09a0-ebc7-410a-86f0-9ed0c04881b9 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103683) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:14:56.995903 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-919f09a0-ebc7-410a-86f0-9ed0c04881b9 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.394084 Oct 13 04:14:56.995903 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 19/39] 10.222.0.27 () {60 vars in 1190 bytes} [Thu Oct 13 04:14:56 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 395 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 13 04:14:57.002600 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-757238e4-54e1-4198-9ceb-81da899526bc admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:57.025209 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-757238e4-54e1-4198-9ceb-81da899526bc admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026546 Oct 13 04:14:57.025760 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 21/40] 10.222.0.27 () {60 vars in 1210 bytes} [Thu Oct 13 04:14:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:14:57.037029 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-b3f2d53f-5967-417c-a99d-76360f4afaf6 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:57.059338 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-b3f2d53f-5967-417c-a99d-76360f4afaf6 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026762 Oct 13 04:14:57.060999 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 20/41] 10.222.0.27 () {60 vars in 1210 bytes} [Thu Oct 13 04:14:57 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:14:57.070557 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-e70dfb78-dc75-4425-ad24-caa2de4288e8 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:57.076650 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.wsgi [None req-e70dfb78-dc75-4425-ad24-caa2de4288e8 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-e70dfb78-dc75-4425-ad24-caa2de4288e8 admin admin] Request method failure captured: Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Accept: application/json Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Accept-Encoding: gzip, deflate Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Connection: keep-alive Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Content-Length: 0 Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Host: 10.222.0.27 Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Openstack-Api-Version: compute 2.92 Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Openstack-System-Scope: None Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: User-Agent: python-novaclient Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Auth-Token: gAAAAABjR5DAZ1RLSglenAik29sKRsFKKqTbkGbzwTdOi6Y9qHzoNxzSjDXVLgse08fLPvEvEavCOWnayoJtF5EUFt4h84DDg7wCopX-9DE3dHtkksaE5jsqX3xsUh8hAyf_xfxMtan3Ppz0DgacsXz57MTWw6l5v-3X5ioRxjeNq7_lJNEKO0I Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Domain-Id: None Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Domain-Name: None Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Identity-Status: Confirmed Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Is-Admin-Project: True Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Openstack-Nova-Api-Version: 2.92 Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Domain-Id: default Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Domain-Name: Default Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Name: admin Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Role: member,admin,reader Oct 13 04:14:57.079904 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Roles: member,admin,reader Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8ba823f313144da3b683bacef8fd62e0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant: admin Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant-Name: admin Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User: admin Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Domain-Id: default Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Domain-Name: Default Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Id: 0e2d979563d6421995451a0e86bf3022 Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Name: admin Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: method: > Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: exception: Flavor 42 could not be found. Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: action_args: {'id': '42'} Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:14:57.081482 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:57.082409 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:57.083490 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-e70dfb78-dc75-4425-ad24-caa2de4288e8 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103684) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:14:57.085134 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-e70dfb78-dc75-4425-ad24-caa2de4288e8 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014618 Oct 13 04:14:57.085134 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 22/42] 10.222.0.27 () {60 vars in 1190 bytes} [Thu Oct 13 04:14:57 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 13 04:14:58.297349 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-b3f2d53f-5967-417c-a99d-76360f4afaf6 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:58.298406 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-b3f2d53f-5967-417c-a99d-76360f4afaf6 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001430 Oct 13 04:14:58.298804 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 21/43] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:14:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:14:58.822772 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-f2697659-7781-4bf8-bd68-841c965ebfcd admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:58.828364 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.wsgi [None req-f2697659-7781-4bf8-bd68-841c965ebfcd admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-f2697659-7781-4bf8-bd68-841c965ebfcd admin admin] Request method failure captured: Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Accept: application/json Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Accept-Encoding: gzip, deflate Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Connection: keep-alive Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Content-Length: 0 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Host: 10.222.0.27 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Openstack-Api-Version: compute 2.92 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Openstack-System-Scope: None Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: User-Agent: python-novaclient Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Auth-Token: gAAAAABjR5DC_0TuOulGNghx_f2v3jctX-6TE8peQ3j0QRPv1xnI6ycuhvC4uPkpiO5a7B8iXopIyKevC9XZKgXR80oOP8LMvN9PzUZgk1PHvLv6opKTfek0vtKF9shHbBFTl9jhuRVRo4iHqmhH6kuo9YcHtajwcfAml62ZiY8NYzCFjFNUgA0 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Domain-Id: None Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Domain-Name: None Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Identity-Status: Confirmed Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Is-Admin-Project: True Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Openstack-Nova-Api-Version: 2.92 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Domain-Id: default Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Domain-Name: Default Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Name: admin Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Role: member,admin,reader Oct 13 04:14:58.829859 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Roles: member,admin,reader Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8ba823f313144da3b683bacef8fd62e0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant: admin Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant-Name: admin Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User: admin Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Domain-Id: default Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Domain-Name: Default Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Id: 0e2d979563d6421995451a0e86bf3022 Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Name: admin Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: method: > Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: exception: Flavor 84 could not be found. Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: action_args: {'id': '84'} Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:14:58.831208 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:58.832143 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.833529 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-f2697659-7781-4bf8-bd68-841c965ebfcd admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103684) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:14:58.834893 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-f2697659-7781-4bf8-bd68-841c965ebfcd admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.396486 Oct 13 04:14:58.834893 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 23/44] 10.222.0.27 () {60 vars in 1190 bytes} [Thu Oct 13 04:14:58 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 398 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 13 04:14:58.841743 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-ca9d8310-de00-4b8b-bbe1-1631e8dbb1b6 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:58.856562 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-ca9d8310-de00-4b8b-bbe1-1631e8dbb1b6 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019168 Oct 13 04:14:58.857085 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 22/45] 10.222.0.27 () {60 vars in 1210 bytes} [Thu Oct 13 04:14:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:14:58.866771 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-4ef13436-5ccd-45a5-8bb7-fdb0c2ff9c98 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:58.882019 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-4ef13436-5ccd-45a5-8bb7-fdb0c2ff9c98 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019020 Oct 13 04:14:58.882587 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1210 bytes} [Thu Oct 13 04:14:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:14:58.892415 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-d3e64de2-8d60-4177-b923-73f2d5530c07 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:14:58.897304 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.wsgi [None req-d3e64de2-8d60-4177-b923-73f2d5530c07 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-d3e64de2-8d60-4177-b923-73f2d5530c07 admin admin] Request method failure captured: Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Accept: application/json Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Accept-Encoding: gzip, deflate Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Connection: keep-alive Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Content-Length: 0 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Host: 10.222.0.27 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Openstack-Api-Version: compute 2.92 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Openstack-System-Scope: None Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: User-Agent: python-novaclient Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Auth-Token: gAAAAABjR5DC_0TuOulGNghx_f2v3jctX-6TE8peQ3j0QRPv1xnI6ycuhvC4uPkpiO5a7B8iXopIyKevC9XZKgXR80oOP8LMvN9PzUZgk1PHvLv6opKTfek0vtKF9shHbBFTl9jhuRVRo4iHqmhH6kuo9YcHtajwcfAml62ZiY8NYzCFjFNUgA0 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Domain-Id: None Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Domain-Name: None Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Identity-Status: Confirmed Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Is-Admin-Project: True Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Openstack-Nova-Api-Version: 2.92 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Domain-Id: default Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Domain-Name: Default Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Name: admin Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Role: member,admin,reader Oct 13 04:14:58.898840 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Roles: member,admin,reader Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8ba823f313144da3b683bacef8fd62e0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant: admin Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant-Name: admin Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User: admin Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Domain-Id: default Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Domain-Name: Default Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Id: 0e2d979563d6421995451a0e86bf3022 Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Name: admin Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: method: > Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: exception: Flavor 84 could not be found. Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: action_args: {'id': '84'} Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:14:58.900333 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:58.901261 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:14:58.902571 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-d3e64de2-8d60-4177-b923-73f2d5530c07 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103683) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:14:58.904362 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-d3e64de2-8d60-4177-b923-73f2d5530c07 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011377 Oct 13 04:14:58.904362 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 23/47] 10.222.0.27 () {60 vars in 1190 bytes} [Thu Oct 13 04:14:58 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 13 04:15:00.107251 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-4ef13436-5ccd-45a5-8bb7-fdb0c2ff9c98 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:00.108271 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-4ef13436-5ccd-45a5-8bb7-fdb0c2ff9c98 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001731 Oct 13 04:15:00.108673 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 25/48] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:15:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:15:00.630652 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-c6a7fce8-c7a4-4716-ac77-4249d34bfc7c admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:00.636439 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.wsgi [None req-c6a7fce8-c7a4-4716-ac77-4249d34bfc7c admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-c6a7fce8-c7a4-4716-ac77-4249d34bfc7c admin admin] Request method failure captured: Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Accept: application/json Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Accept-Encoding: gzip, deflate Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Connection: keep-alive Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Content-Length: 0 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Host: 10.222.0.27 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Openstack-Api-Version: compute 2.92 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Openstack-System-Scope: None Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: User-Agent: python-novaclient Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Auth-Token: gAAAAABjR5DEBCjwetcLJN7RKEk7LET7Nj5HJgLPbxXGB-u-W8zs5NVMvaUsSwwRzT52LfPcfx6gNH0QFiq_n7fZamNgvIyN-dTKqIRd5NygNMT7TFfpg5222hdY4vcueE-nb_A5cz5H4joPdDPyXjXU-RLxU5JmKpCdHYBCjrkFZCNMOgKPvm4 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Domain-Id: None Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Domain-Name: None Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Identity-Status: Confirmed Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Is-Admin-Project: True Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Openstack-Nova-Api-Version: 2.92 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Domain-Id: default Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Domain-Name: Default Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Project-Name: admin Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Role: member,admin,reader Oct 13 04:15:00.638120 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Roles: member,admin,reader Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8ba823f313144da3b683bacef8fd62e0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant: admin Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-Tenant-Name: admin Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User: admin Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Domain-Id: default Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Domain-Name: Default Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Id: 0e2d979563d6421995451a0e86bf3022 Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: X-User-Name: admin Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: method: > Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: exception: Flavor 451 could not be found. Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: action_args: {'id': '451'} Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:15:00.639623 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:15:00.640395 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.641392 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-c6a7fce8-c7a4-4716-ac77-4249d34bfc7c admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103683) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:15:00.642496 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-c6a7fce8-c7a4-4716-ac77-4249d34bfc7c admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.394388 Oct 13 04:15:00.642496 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 24/49] 10.222.0.27 () {60 vars in 1193 bytes} [Thu Oct 13 04:15:00 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 395 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 13 04:15:00.649303 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-420248b6-f0ba-4980-a084-853744c5ec03 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:00.665406 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-420248b6-f0ba-4980-a084-853744c5ec03 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019667 Oct 13 04:15:00.665949 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 26/50] 10.222.0.27 () {60 vars in 1210 bytes} [Thu Oct 13 04:15:00 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:15:00.676371 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:00.691195 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018426 Oct 13 04:15:00.691735 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 25/51] 10.222.0.27 () {60 vars in 1210 bytes} [Thu Oct 13 04:15:00 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:15:00.701023 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-942db144-2f29-405f-bf99-a70aa7a67794 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:00.705871 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.wsgi [None req-942db144-2f29-405f-bf99-a70aa7a67794 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-942db144-2f29-405f-bf99-a70aa7a67794 admin admin] Request method failure captured: Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Accept: application/json Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Accept-Encoding: gzip, deflate Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Connection: keep-alive Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Content-Length: 0 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Host: 10.222.0.27 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Openstack-Api-Version: compute 2.92 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Openstack-System-Scope: None Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: User-Agent: python-novaclient Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Auth-Token: gAAAAABjR5DEBCjwetcLJN7RKEk7LET7Nj5HJgLPbxXGB-u-W8zs5NVMvaUsSwwRzT52LfPcfx6gNH0QFiq_n7fZamNgvIyN-dTKqIRd5NygNMT7TFfpg5222hdY4vcueE-nb_A5cz5H4joPdDPyXjXU-RLxU5JmKpCdHYBCjrkFZCNMOgKPvm4 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Domain-Id: None Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Domain-Name: None Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Identity-Status: Confirmed Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Is-Admin-Project: True Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Openstack-Nova-Api-Version: 2.92 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Domain-Id: default Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Domain-Name: Default Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Project-Name: admin Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Role: member,admin,reader Oct 13 04:15:00.707423 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Roles: member,admin,reader Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8ba823f313144da3b683bacef8fd62e0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8ba823f313144da3b683bacef8fd62e0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant: admin Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant-Id: 8ba823f313144da3b683bacef8fd62e0 Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-Tenant-Name: admin Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User: admin Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Domain-Id: default Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Domain-Name: Default Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Id: 0e2d979563d6421995451a0e86bf3022 Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: X-User-Name: admin Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: method: > Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: exception: Flavor 451 could not be found. Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: action_args: {'id': '451'} Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:15:00.708944 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:15:00.709778 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: ERROR nova.api.openstack.wsgi Oct 13 04:15:00.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-942db144-2f29-405f-bf99-a70aa7a67794 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103684) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:15:00.711910 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-942db144-2f29-405f-bf99-a70aa7a67794 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011511 Oct 13 04:15:00.711910 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 27/52] 10.222.0.27 () {60 vars in 1193 bytes} [Thu Oct 13 04:15:00 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 13 04:15:01.924755 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:01.925785 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Oct 13 04:15:01.926180 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 26/53] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:15:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:15:02.449535 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-e27f9437-c8cc-4b80-8dda-a9f5e2b0d518 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:15:02.468278 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-e27f9437-c8cc-4b80-8dda-a9f5e2b0d518 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.403324 Oct 13 04:15:02.468823 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 28/54] 10.222.0.27 () {64 vars in 1235 bytes} [Thu Oct 13 04:15:02 2022] POST /compute/v2.1/flavors => generated 404 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:15:03.693699 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:03.694730 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Oct 13 04:15:03.695127 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 27/55] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:15:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:15:04.218295 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-4e041c78-eb0b-4bdc-80a5-480602736f31 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:15:04.237060 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-4e041c78-eb0b-4bdc-80a5-480602736f31 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.405555 Oct 13 04:15:04.237609 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 29/56] 10.222.0.27 () {64 vars in 1235 bytes} [Thu Oct 13 04:15:03 2022] POST /compute/v2.1/flavors => generated 405 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:15:05.462900 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:05.463941 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Oct 13 04:15:05.464369 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 28/57] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:15:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:15:05.982066 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-a504f013-6581-4309-a90f-f49aa9d265fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:15:06.001251 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-a504f013-6581-4309-a90f-f49aa9d265fd admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.398806 Oct 13 04:15:06.001784 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 30/58] 10.222.0.27 () {64 vars in 1235 bytes} [Thu Oct 13 04:15:05 2022] POST /compute/v2.1/flavors => generated 407 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:15:07.224540 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:07.225565 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Oct 13 04:15:07.225955 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 29/59] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:15:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:15:07.748230 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-e4465723-87a9-4927-a8d5-e271113d1a7f admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:15:07.766980 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-e4465723-87a9-4927-a8d5-e271113d1a7f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.402683 Oct 13 04:15:07.767533 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 31/60] 10.222.0.27 () {64 vars in 1235 bytes} [Thu Oct 13 04:15:07 2022] POST /compute/v2.1/flavors => generated 409 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:15:08.978121 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:08.979146 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001715 Oct 13 04:15:08.979541 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 30/61] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:15:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:15:09.496567 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-bb183c86-dcb2-4dff-9f3a-3ae1fd0c87eb admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:15:09.518407 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-bb183c86-dcb2-4dff-9f3a-3ae1fd0c87eb admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.402624 Oct 13 04:15:09.518976 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 32/62] 10.222.0.27 () {60 vars in 1202 bytes} [Thu Oct 13 04:15:09 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:16:06.337462 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:16:06.338528 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-1cfebbba-aa38-49c7-a92d-e85c08b197c3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Oct 13 04:16:06.338958 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 31/63] 10.222.0.27 () {54 vars in 931 bytes} [Thu Oct 13 04:16:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:16:06.350233 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:16:06.729396 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-cfcae5c4-7c44-41c9-9ae9-f4c8e1dd3b11 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:16:06.749881 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-cfcae5c4-7c44-41c9-9ae9-f4c8e1dd3b11 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.404136 Oct 13 04:16:06.750442 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 33/64] 10.222.0.27 () {60 vars in 1292 bytes} [Thu Oct 13 04:16:06 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:16:06.771270 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_api:SELECT=14 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:16:06.775113 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-8f3a373c-58e4-47ff-b0aa-c6ac84be1bc5 admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:16:06.782621 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-8f3a373c-58e4-47ff-b0aa-c6ac84be1bc5 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012755 Oct 13 04:16:06.783151 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 32/65] 10.222.0.27 () {60 vars in 1283 bytes} [Thu Oct 13 04:16:06 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:16:06.794976 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-86e6e26a-80e5-4ecd-b17d-931a46d90234 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:16:06.802195 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-86e6e26a-80e5-4ecd-b17d-931a46d90234 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011130 Oct 13 04:16:06.802734 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 34/66] 10.222.0.27 () {60 vars in 1286 bytes} [Thu Oct 13 04:16:06 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:16:06.814305 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:16:06.821270 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010387 Oct 13 04:16:06.821779 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 33/67] 10.222.0.27 () {60 vars in 1286 bytes} [Thu Oct 13 04:16:06 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 11 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:16:06.833323 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-49cd55ef-db56-4de6-bdc7-fbdb24158e12 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:16:06.840573 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-49cd55ef-db56-4de6-bdc7-fbdb24158e12 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010723 Oct 13 04:16:06.841085 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 35/68] 10.222.0.27 () {60 vars in 1283 bytes} [Thu Oct 13 04:16:06 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:18:58.937245 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:18:58.938358 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001538 Oct 13 04:18:58.938753 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 34/69] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:18:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:18:59.079336 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_api:SELECT=6 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:18:59.242219 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:18:59.246960 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:18:59.247425 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:18:59.247858 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:18:59.248579 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:18:59.249028 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:18:59.249450 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:18:59.280603 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-37bea96d-cccd-4365-96c2-d6bf7989aba4 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.202546 Oct 13 04:18:59.281166 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 36/70] 10.222.0.27 () {60 vars in 1193 bytes} [Thu Oct 13 04:18:59 2022] GET /compute/v2.1/os-services => generated 791 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:19:00.357370 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:19:00.358411 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 Oct 13 04:19:00.358818 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 35/71] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:19:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:19:00.549192 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:19:00.552595 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:19:00.553079 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:19:00.553513 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:19:00.554281 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:19:00.554732 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:19:00.555151 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:19:00.576434 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-e89f8510-943a-45c2-af92-faeb3c322c9b admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.078602 Oct 13 04:19:00.576981 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 37/72] 10.222.0.27 () {60 vars in 1193 bytes} [Thu Oct 13 04:19:00 2022] GET /compute/v2.1/os-services => generated 791 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:19:12.058294 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:19:12.059372 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001449 Oct 13 04:19:12.059765 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 36/73] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:19:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:19:12.202918 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_cell0:SELECT=4 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:19:12.203627 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_cell1:SELECT=4 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:19:12.204318 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG dbcounter [-] [103684] Writing DB stats nova_api:SELECT=8 {{(pid=103684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:19:12.253319 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:19:12.256747 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:19:12.257218 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:19:12.257639 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:19:12.258380 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:19:12.258835 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:19:12.259233 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:19:12.284530 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-fa7d838c-d593-4c3d-b0da-6b74398d86d2 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.086037 Oct 13 04:19:12.285003 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 38/74] 10.222.0.27 () {60 vars in 1193 bytes} [Thu Oct 13 04:19:12 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:19:13.352598 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG nova.api.openstack.wsgi [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] Calling method '>' {{(pid=103683) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:19:13.353622 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: INFO nova.api.openstack.requestlog [None req-4d38ceae-c5d8-4298-a259-a2ce634e473f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001418 Oct 13 04:19:13.354021 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: [pid: 103683|app: 0|req: 37/75] 10.222.0.27 () {56 vars in 1074 bytes} [Thu Oct 13 04:19:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:19:13.544968 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG nova.api.openstack.wsgi [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Calling method '>' {{(pid=103684) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:19:13.548322 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:19:13.548794 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:19:13.549217 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:19:13.550141 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:19:13.550614 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:19:13.551027 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: DEBUG oslo_concurrency.lockutils [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103684) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:19:13.572687 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: INFO nova.api.openstack.requestlog [None req-a3b50647-a81d-4071-b5fc-e9b1292e24e8 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.078720 Oct 13 04:19:13.573165 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: [pid: 103684|app: 0|req: 39/76] 10.222.0.27 () {60 vars in 1193 bytes} [Thu Oct 13 04:19:13 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:20:07.893118 dv-cinder-860997-1-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 13 04:20:07.893664 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: Thu Oct 13 04:20:07 2022 - graceful shutdown triggered... Oct 13 04:20:07.894152 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: Gracefully killing worker 1 (pid: 103683)... Oct 13 04:20:07.894551 dv-cinder-860997-1-iscsi devstack@n-api.service[103684]: Gracefully killing worker 2 (pid: 103684)... Oct 13 04:20:08.019413 dv-cinder-860997-1-iscsi devstack@n-api.service[103683]: DEBUG dbcounter [-] [103683] Writing DB stats nova_api:SELECT=4 {{(pid=103683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:08.894332 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: worker 1 buried after 1 seconds Oct 13 04:20:08.894858 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: worker 2 buried after 1 seconds Oct 13 04:20:08.894858 dv-cinder-860997-1-iscsi devstack@n-api.service[103682]: goodbye to uWSGI. Oct 13 04:20:08.895951 dv-cinder-860997-1-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 13 04:20:08.896511 dv-cinder-860997-1-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 13 04:20:08.899319 dv-cinder-860997-1-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 13 04:20:08.909112 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 13 04:20:08 2022] *** Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: nodename: dv-cinder-860997-1-iscsi Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: machine: x86_64 Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: clock source: unix Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: pcre jit disabled Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: detected number of CPU cores: 4 Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: current working directory: / Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: detected binary path: /usr/bin/uwsgi-core Oct 13 04:20:08.911120 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: your processes number limit is 47680 Oct 13 04:20:08.911971 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: your memory page size is 4096 bytes Oct 13 04:20:08.911971 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: detected max file descriptor number: 2048 Oct 13 04:20:08.911971 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: lock engine: pthread robust mutexes Oct 13 04:20:08.911971 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: thunder lock: enabled Oct 13 04:20:08.911971 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 13 04:20:08.911971 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 13 04:20:08.951203 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: Python main interpreter initialized at 0x55787c5c69c0 Oct 13 04:20:08.951203 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: python threads support enabled Oct 13 04:20:08.951203 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: your server socket listen backlog is limited to 100 connections Oct 13 04:20:08.951203 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: your mercy for graceful operations on workers is 90 seconds Oct 13 04:20:08.951607 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: mapped 403077 bytes (393 KB) for 2 cores Oct 13 04:20:08.951677 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: *** Operational MODE: preforking *** Oct 13 04:20:08.951912 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: *** uWSGI is running in multiple interpreter mode *** Oct 13 04:20:08.951912 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: spawned uWSGI master process (pid: 118657) Oct 13 04:20:08.958279 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: spawned uWSGI worker 1 (pid: 118673, cores: 1) Oct 13 04:20:08.958279 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: spawned uWSGI worker 2 (pid: 118674, cores: 1) Oct 13 04:20:08.958279 dv-cinder-860997-1-iscsi devstack@n-api.service[118657]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 13 04:20:08.952124 dv-cinder-860997-1-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 13 04:20:09.623378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 13 04:20:09.630990 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 13 04:20:10.806001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:20:10.806001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: from cryptography.utils import int_from_bytes Oct 13 04:20:10.806815 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:20:10.806815 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: from cryptography.utils import int_from_bytes Oct 13 04:20:10.807123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:20:10.807123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: from cryptography.utils import int_from_bytes Oct 13 04:20:10.807590 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 04:20:10.807590 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: from cryptography.utils import int_from_bytes Oct 13 04:20:13.252584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 13 04:20:13.253171 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 13 04:20:13.253384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 13 04:20:13.253748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 13 04:20:13.254010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 13 04:20:13.254427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.254761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.255086 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.255408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.255740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.256054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.256377 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.256697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.257028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.257289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.257606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.258072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860997-1-iscsi {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.258432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.258745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.259060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.259381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.259481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 13 04:20:13.259856 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.259956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 13 04:20:13.260040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.260231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 13 04:20:13.260323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.260593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 13 04:20:13.260778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.260889 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 13 04:20:13.261225 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.261341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.261535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.261648 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.261869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.261978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.262297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.262424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.262581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.262697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.262881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263191 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.263945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.264143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.264248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.264534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.264639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860997-1-iscsi {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.264866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.264967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860997-1-iscsi {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.265168 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.265275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.265475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.266361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.266564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.266740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.266892 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267166 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267429 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267548 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.267874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.268186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.268328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.268532 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.268672 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.268840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.268934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.269149 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.269240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.269469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.269577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.269775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.269873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.270078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.270174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.270412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.270521 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.270717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.270813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271029 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271119 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860997-1-iscsi {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.271891 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.272041 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.272233 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.272382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.272528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.272627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.272851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273440 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273649 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.273967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.274065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.274297 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.274409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.274603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.274693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.274911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275840 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.275938 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.276136 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.276243 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.276441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.276542 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.276744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.276843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277156 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277660 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277751 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277907 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.277998 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.278232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.278397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.278553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.278644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.278868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.278961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279186 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279580 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.279989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.280210 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.280300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.280532 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.280631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.280842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.280942 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.281148 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.281300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.282012 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.282416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.282628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.282770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.282770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.282936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283279 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283404 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283596 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.283911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284013 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284533 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.284950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.285209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.285306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.285477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.285578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.285781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.285889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.286098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.286190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.286436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.286571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.286738 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.286845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287450 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.287956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.288047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.288264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.288353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.288571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.288674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.288892 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289288 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.289868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290123 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.290966 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.291966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.292170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.292268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.292476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.292617 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.292789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.292896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293380 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.293987 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.294098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.294312 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.294447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.294617 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.294772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.294933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295554 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295860 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.295958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.296161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.296259 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.296462 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.296568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.296769 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.296903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.297101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.297266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.297423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.297594 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.297724 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.297850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.298034 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.298169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.298375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.298583 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.298675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.298908 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.299052 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.299217 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.299361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.299524 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.299615 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.299840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.299960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.300142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.300277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.300448 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.300612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.300763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.300943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.301067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.301261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.301361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.301575 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.301665 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.301891 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.301996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.302245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.302354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.302569 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.302714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.302847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.302948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.303147 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.303245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.303443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.303541 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.303759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.303858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.304075 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.304165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.304381 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.304522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.304688 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.304787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.304993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.305084 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.305302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.305392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 04:20:13.305622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.305716 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.305949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.306091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.306300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.306392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.306615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.306718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.306934 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307690 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.307977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.308203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.308300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.308518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.308620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.308826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.308916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.309135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.309230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.309444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.309547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.309760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.309853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.310077 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.310167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.310426 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.310517 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.310738 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.310835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311053 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311149 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311464 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.311987 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.312076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.312301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.312393 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.312801 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.312964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.313131 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.313250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.313442 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.313545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.313767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.313869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.314081 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.314179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.314429 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.314519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.314745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.314837 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.315977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.316079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.316286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.316387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.316593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.316686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.316907 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317844 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.317976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.318160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.318313 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.318494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.318681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.318802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.318904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.319113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.319258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.319426 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.319564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.319740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.319832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320138 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320360 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.320977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321284 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.321996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.322238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.322348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.322522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.322625 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.322826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.322923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.323134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.323281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.323436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.323566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.323748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.323877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324172 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324458 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.324993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.325096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.325302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.325392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.325609 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.325697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.325917 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326549 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.326950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.327163 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.327256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.327485 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.327577 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.327795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.327884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.328622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.328764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.328935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.329217 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.329416 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.329545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.329645 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.329853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.329944 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.330159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.330265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.330492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.330598 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.330800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.330899 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.331109 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.331210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.331420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.331509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.331733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.331831 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332141 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332346 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332653 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.332994 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.333299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.333397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.333619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.333723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.333947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.334037 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.334286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.334391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.334610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.334763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.334902 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335192 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335806 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.335918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.336114 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.336214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.336417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.336598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.336715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.336799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337127 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337309 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337598 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.337918 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338856 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.338954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.339159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.339258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.339499 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.339589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.339803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.339900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.340139 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.340232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.340464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.340564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.340778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.340886 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.341096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.341188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.341404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.341492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.341713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.341846 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342022 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342355 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342664 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.342972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.343105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.343288 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.343377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.343594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.343698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.344491 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.344752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.344925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.345066 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.345159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.345391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.345485 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.345698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.345796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346101 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346343 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346437 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.346996 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.347096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.347312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.347407 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.347630 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.347761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.347940 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348560 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348867 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.348970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.349171 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.349260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.349479 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.349620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.349793 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.349883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.350099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.350195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.350438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.350543 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.350744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.350854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351148 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351364 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.351968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.352064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.352280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.352376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.352603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.352727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.352914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.353013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.353229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.353327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354834 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.354924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.355014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.355098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.355188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.355346 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.355445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.355652 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356217 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356319 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356617 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356839 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.356970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.357143 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.357241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.357459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.357550 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.357770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.357903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.358089 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.358451 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.358546 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.358766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.358857 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.359087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.359196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.359435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.359578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.359781 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.359953 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360279 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360452 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360646 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360739 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.360960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.361064 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.361286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.361377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.361599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.361699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.361914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.362978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.363177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.363276 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.363482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.363763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.363865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.364099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.364190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.364410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.364528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.364722 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.364861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365031 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365340 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.365983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.366111 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.366297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.366416 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.366608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.366702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.366921 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367648 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.367992 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.368162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.368287 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.368469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.368566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.368774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.368944 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369470 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369827 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.369990 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.370266 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.370373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.370605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.370709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.370918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371015 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.371967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.372173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.372277 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.372479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.372571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.372786 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.372876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.373092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.373218 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.373399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.373529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.373705 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.373846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374646 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.374951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375472 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375633 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375717 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375900 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.375995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.376202 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.376292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.376503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.376607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.376805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.376904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.377118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.377218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.377422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.377522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.377732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.377831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.378510 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.378620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.378761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.378868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.378961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379211 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379549 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379649 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.379961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.380174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.380273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.380480 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.380579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.380784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.380883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.381097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.381189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.381407 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.381498 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.381713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.381803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382122 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.382954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383350 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383666 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.383957 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.384177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.384267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.384481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.384584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.384785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.384875 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385093 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385787 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.385996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.386093 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.386326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.386432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.386634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.386733 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.386933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387325 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.387936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.388154 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.388294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.388459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.388559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.388759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.388885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.389958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.390056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.390289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.390394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.391167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.391365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.391524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.391694 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.391795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.391880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.392980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.393079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.393753 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.393887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394000 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.394909 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395084 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395690 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395789 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.395990 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.396134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.396296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.396422 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.396638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.396732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.396940 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397240 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397659 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.397974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.398159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.398320 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.398495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.398621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.398796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.398937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.399099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.399193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.399400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.399491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.399734 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.399832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400166 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400658 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.400971 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.401063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.401285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.401385 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.401594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.401691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.401906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402005 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402269 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.402940 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.403155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.403253 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.403464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.403563 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.403767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.403907 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.404071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.404162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.404390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.404495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.404699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.404789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405313 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405722 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.405933 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.406069 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.406930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.407127 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.407336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.407541 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.407680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.407793 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.407917 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408375 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408691 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.408811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409744 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.409935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.410053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.410626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.410769 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.410893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411012 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411249 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411483 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411721 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.411948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.412057 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.412324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.412452 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.412572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.412683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.412793 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.413057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.413185 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.413382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.413509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.413689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.413935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.414083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.414223 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.414450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.414570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.414771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.414898 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.415081 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.415200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.415388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.415516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.415701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.415819 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.416530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.416663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.416799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.416925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417060 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417575 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.417977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418381 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418507 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418645 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.418903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419535 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419889 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.419987 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420084 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420569 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.420986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.421154 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.421300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.421484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.421592 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.421806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.422055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.422356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.422477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.422690 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.422800 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423731 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.423941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.424981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.425202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.425295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.425510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.425601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.425819 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.425921 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.426122 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.426240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.426469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.426563 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.426792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.426883 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.427097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.427195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.427423 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.427527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.427741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.427833 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.428068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.428168 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.428394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.428495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.428703 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.428803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429122 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429345 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429781 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.429960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.430253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.430354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.430567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.430706 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.430876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.430968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.431201 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.431324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.431508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.431614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.431822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.431921 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.432134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.432234 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.432444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.432535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.432752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.432850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.433056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.433205 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.433392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.433522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.433710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.433806 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.434031 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.434134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.434382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.434477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.434690 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.434782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435110 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.435918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.436911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.437121 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.437219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.437443 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.438115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.438344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.438546 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.438641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.438950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439567 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.439972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.440216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.440310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.440522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.440614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.440852 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.440945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.441163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.441257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.441490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.441582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.441807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.441901 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.442117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.442238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.442457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.442552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.442766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.442867 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443075 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443470 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443690 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443791 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.443995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.444095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.444299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.444392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.444600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.444700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.444926 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.445255 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.445491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.445585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.445825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.445928 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.446144 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.446257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.446493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.446636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.446797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.446889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.447101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.447196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.447415 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.447513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.447733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.447827 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.448050 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.448151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.448362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.448460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.448674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.448767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449093 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.449954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.450053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.450292 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.450438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.450609 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.450702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.450926 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451234 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.451961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.452188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.452281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.452500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.452736 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.452834 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.453160 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.453844 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.454052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.454301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.454494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.454615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.454744 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.454931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455072 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455555 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.455967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.456174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.456312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.456487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.456593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.456813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.456916 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.457134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.457233 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.457455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.457565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.457773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.457871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.458090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.458191 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.458429 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.458522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.458750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.458950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.459270 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.459372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.459606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.459709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.459935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460027 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460254 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460573 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.460980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.461201 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.461307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.461509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.461606 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.461818 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.461911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.462129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.462268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.462465 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.462557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.462775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.462874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.463082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.463186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.463395 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.463495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.463702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.463801 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464124 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.464972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.465222 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.465317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.465545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.465640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.465872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.466078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.466221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.466417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.466528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.466748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.466847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.467073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.467166 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.467397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.467488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.467712 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.467812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468338 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468643 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.468979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469693 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.469982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.470193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.470309 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.470522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.470673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.470854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.470982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.471159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.471271 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.471468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.471561 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.471776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.472980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.473071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.473287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.473389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.473601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.473702 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.473909 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.474009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.474256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.474393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.474565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.474692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.474935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.475126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.475297 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.475454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.475610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.475769 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.475921 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.476077 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.476251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.476384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.476561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.476697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.476868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477827 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.477919 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.478142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.478293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.478482 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.478577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.478828 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.478928 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.479146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.479237 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.479458 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.479679 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.479779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.480017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.480112 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.480337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.480428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.480640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481019 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481204 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481507 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481830 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.481921 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.482138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.482262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.482493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.482586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.482807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.482912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.483120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.483213 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.483434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.483532 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.483749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.483850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.484050 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.484148 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.484378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.484502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.484708 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.484892 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485413 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485630 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.485940 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.486032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.486307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.486417 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.486625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.486718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.486962 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.487160 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.487281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.487496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.487624 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.487818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.487943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.488152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.488246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.488487 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.488587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.488802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.488902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.489108 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.489200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.489414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.489512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.489739 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.489831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.490051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.490143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.490389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.490485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.490699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.490800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491624 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.491936 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492246 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.492966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.493175 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.493267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.493484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.493581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.493791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.493896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.494101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.494194 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.494439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.494532 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.494778 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.494916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.495096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.495223 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.495414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.495514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.495731 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.495823 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.496968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497058 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497277 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497370 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.497982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.498195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.498374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.498526 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.498618 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.498832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.498949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.499138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.499262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.499447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.499581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.499753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.499899 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.500585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.500745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.500960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.501137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.501290 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.501515 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.501646 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.501830 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.501926 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.502137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.502248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.502472 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.502578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.502783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.502884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.503088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.503180 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.503394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.503494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.503701 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.503794 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504007 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.504927 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505243 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505853 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.505944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.506155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.506315 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.506489 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.506581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.506792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.506884 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.507095 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.507230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.507397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.507496 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.507699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.507810 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508099 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.508996 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.509208 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.509305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.509510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.509681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.509811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.509912 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.510120 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.510231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.510448 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.510552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.510752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.510845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511657 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511756 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.511988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.512087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.512319 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.512418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.512632 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.512731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.512954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513573 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.513983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.514190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.514301 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.514533 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.514631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.514844 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.514943 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.515153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.515244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.515476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.515569 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.515963 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.516151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.516289 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.516481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.516582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.516816 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.516916 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.517132 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.517234 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.517463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.517558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.517786 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.517891 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.518110 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.518233 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.518441 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.518534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.518766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.518870 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.519086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.519188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.519411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.519504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.519725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.519830 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.520035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.520129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.520281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 13 04:20:13.520374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.520656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.520967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.521275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.521586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.521895 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.522225 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.522540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.522850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.523165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.523482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.523807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.524145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.524437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.524759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.525082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.525425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.525735 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.526075 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.526424 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.526738 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.527049 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.527352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 04:20:13.527592 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118674) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 13 04:20:13.583585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO dbcounter [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Registered counter for database nova_api Oct 13 04:20:13.590351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO dbcounter [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Registered counter for database nova_api Oct 13 04:20:13.604773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_db.sqlalchemy.engines [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118673) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.613321 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_db.sqlalchemy.engines [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118674) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.613604 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writer thread running {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.619951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writer thread running {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.669864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.context [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),1311a648-eb92-4594-8c2f-3c55fab1c340(cell1) {{(pid=118673) load_cells /opt/stack/nova/nova/context.py:464}} Oct 13 04:20:13.674597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:20:13.675074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:20:13.676569 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.context [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),1311a648-eb92-4594-8c2f-3c55fab1c340(cell1) {{(pid=118674) load_cells /opt/stack/nova/nova/context.py:464}} Oct 13 04:20:13.676697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:20:13.677387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:20:13.677830 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:20:13.679873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:20:13.680575 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:20:13.681046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:20:13.681883 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO dbcounter [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Registered counter for database nova_cell0 Oct 13 04:20:13.682629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:20:13.683359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:20:13.683812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:20:13.685922 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:20:13.686047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO dbcounter [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Registered counter for database nova_cell1 Oct 13 04:20:13.688192 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO dbcounter [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Registered counter for database nova_cell0 Oct 13 04:20:13.692190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO dbcounter [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Registered counter for database nova_cell1 Oct 13 04:20:13.693328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_db.sqlalchemy.engines [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118673) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.698662 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writer thread running {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.699332 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_db.sqlalchemy.engines [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118673) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.701329 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_db.sqlalchemy.engines [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118674) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.702495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_db.sqlalchemy.engines [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118674) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.707154 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writer thread running {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.844810 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writer thread running {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.853405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writer thread running {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.854055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO dbcounter [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Registered counter for database nova_cell0 Oct 13 04:20:13.864211 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_db.sqlalchemy.engines [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118673) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.865648 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO dbcounter [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Registered counter for database nova_cell0 Oct 13 04:20:13.870456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writer thread running {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:13.872814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_db.sqlalchemy.engines [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118674) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 04:20:13.877460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writer thread running {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 04:20:14.196025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: WARNING keystonemiddleware.auth_token [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:20:14.201597 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: WARNING keystonemiddleware.auth_token [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:20:14.203669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: WARNING keystonemiddleware.auth_token [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:20:14.209089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: WARNING keystonemiddleware.auth_token [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:20:14.551180 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: WARNING keystonemiddleware.auth_token [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:20:14.554841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: WARNING keystonemiddleware.auth_token [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 13 04:20:14.557183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: WARNING keystonemiddleware.auth_token [None req-fa02a98f-60fe-4ba0-92c9-e216f41e0e3e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:20:14.560774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: WARNING keystonemiddleware.auth_token [None req-da87dcd3-89cc-49eb-8eba-a1bcfcf7c195 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 13 04:20:14.562663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55787c5c69c0 pid: 118673 (default app) Oct 13 04:20:14.566058 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55787c5c69c0 pid: 118674 (default app) Oct 13 04:20:50.329018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.330508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.331908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.332694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.921523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.922412 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.923552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:50.924577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:20:51.990035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cda81e61-5d43-412a-9fd0-c799786208aa tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Action: 'create', calling method: .version_select at 0x7f07959d04c0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-500091789"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:20:52.603497 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3b0bfd81-00f7-4695-8924-f24694dccec6 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Action: 'create', calling method: .version_select at 0x7f07959d2550>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-209420894", "metadata": {"Type": "work"}}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:20:53.864520 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3b0bfd81-00f7-4695-8924-f24694dccec6 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 2.948947 Oct 13 04:20:53.876455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 1/1] 10.222.0.27 () {60 vars in 1156 bytes} [Thu Oct 13 04:20:50 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 2952 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 13 04:20:53.881038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1c9c0d49-4c85-407a-8fb0-762818842766 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Calling method '.version_select at 0x7f07958a7430>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:53.982372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cda81e61-5d43-412a-9fd0-c799786208aa tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 3.661057 Oct 13 04:20:53.983769 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 1/2] 10.222.0.27 () {60 vars in 1155 bytes} [Thu Oct 13 04:20:50 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 3664 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 13 04:20:54.000335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-eff24603-d142-4fe7-83e5-8f543586e17c tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a43a0>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:54.093198 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1c9c0d49-4c85-407a-8fb0-762818842766 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f" status: 200 len: 352 microversion: 2.1 time: 0.216012 Oct 13 04:20:54.094245 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:53 2022] GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f => generated 352 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:54.189256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-eff24603-d142-4fe7-83e5-8f543586e17c tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 200 len: 344 microversion: 2.1 time: 0.193418 Oct 13 04:20:54.197244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:53 2022] GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:20:55.109604 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9ad6a7e2-c437-40d8-9272-7de45513d3d6 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Calling method '.version_select at 0x7f07958a7af0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:55.210540 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e27db7ca-1e61-42c6-883e-8b85abaf20e5 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a4a60>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:55.225773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9ad6a7e2-c437-40d8-9272-7de45513d3d6 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f" status: 200 len: 353 microversion: 2.1 time: 0.121444 Oct 13 04:20:55.227171 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 3/5] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:55 2022] GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f => generated 353 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:55.241128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a5d6d857-054d-4554-bf18-ea22d202cbef tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Calling method '.version_select at 0x7f07958401f0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:55.325412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e27db7ca-1e61-42c6-883e-8b85abaf20e5 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 200 len: 345 microversion: 2.1 time: 0.125055 Oct 13 04:20:55.329268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 3/6] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:55 2022] GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 345 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:55.345293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c2d5b48f-2e79-4fc3-944f-25067c1cfe75 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Action: 'create', calling method: .version_select at 0x7f07958b40d0>, body: {"snapshot": {"volume_id": "0f05c3aa-7c97-4c61-8611-fadb9f7c90e9", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-871712501"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:20:55.437433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a5d6d857-054d-4554-bf18-ea22d202cbef tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f" status: 200 len: 353 microversion: 2.1 time: 0.199346 Oct 13 04:20:55.438564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:55 2022] GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f => generated 353 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:20:55.452631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-36432bde-ec7e-4bf6-bb1c-eee6dc6f8d5c tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Calling method '.version_select at 0x7f07958a7c10>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:55.616347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-36432bde-ec7e-4bf6-bb1c-eee6dc6f8d5c tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f" status: 202 len: 0 microversion: 2.1 time: 0.167757 Oct 13 04:20:55.621111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 5/8] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:20:55 2022] DELETE /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f => generated 0 bytes in 173 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:20:55.627157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a71f510a-477c-4633-8f10-9545fbc94025 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Calling method '.version_select at 0x7f07958a7550>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:56.048218 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a71f510a-477c-4633-8f10-9545fbc94025 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f" status: 200 len: 352 microversion: 2.1 time: 0.425858 Oct 13 04:20:56.048868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 6/9] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:55 2022] GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f => generated 352 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:56.129181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c2d5b48f-2e79-4fc3-944f-25067c1cfe75 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.790205 Oct 13 04:20:56.131128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 4/10] 10.222.0.27 () {60 vars in 1162 bytes} [Thu Oct 13 04:20:55 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:20:56.143281 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-49f6ba8b-f941-4474-aa94-299ecc603abf tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f0795840310>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:56.269030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-49f6ba8b-f941-4474-aa94-299ecc603abf tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 200 len: 286 microversion: 2.1 time: 0.130695 Oct 13 04:20:56.275812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 7/11] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:20:56 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:56.281329 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1a7295d8-aee0-4935-af25-6a6d7f66102e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958b4af0>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:56.355177 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1a7295d8-aee0-4935-af25-6a6d7f66102e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.078171 Oct 13 04:20:56.355741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 5/12] 10.222.0.27 () {58 vars in 1140 bytes} [Thu Oct 13 04:20:56 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:56.368212 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0cdf91a3-0427-459e-9575-fc2adc068f80 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a74c0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:56.422750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0cdf91a3-0427-459e-9575-fc2adc068f80 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 200 len: 286 microversion: 2.1 time: 0.058383 Oct 13 04:20:56.427955 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 8/13] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:20:56 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 286 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:57.080193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cf3a6f48-1b56-42de-9bed-3784483a02d2 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Calling method '.version_select at 0x7f07958a49d0>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:57.143458 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-cf3a6f48-1b56-42de-9bed-3784483a02d2 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] HTTP exception thrown: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cf3a6f48-1b56-42de-9bed-3784483a02d2 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Request method failure captured: Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f HTTP/1.1 Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IirA7cBZ5M9fVKf4MyGpjrBLcgC9AieTirT8PdtgHdBvEP_I2h0CC3SE84js3xVHeYiT6NzMTg2BKgTixxVHumsA9_d6R-DVvfKIQNJYJt0MTD7KaeOlRiWYfvi5J2OKsAFaOymEwsHLpK-PhsQPCexPVgksz0g4gOVZhOMpH70Kc Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: bb1bd95475ad40029bd402081f61a70c Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesGetTestJSON-240553955 Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:20:57.149356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/bb1bd95475ad40029bd402081f61a70c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/bb1bd95475ad40029bd402081f61a70c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/bb1bd95475ad40029bd402081f61a70c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_bb1bd95475ad40029bd402081f61a70c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesGetTestJSON-240553955 Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: bb1bd95475ad40029bd402081f61a70c Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesGetTestJSON-240553955 Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesGetTestJSON-240553955-project Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: c870f42503d0471881b812c024fddb7f Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesGetTestJSON-240553955-project Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '43033a98-054b-499b-9887-890a3a4ca28f'} Oct 13 04:20:57.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. (HTTP 404) (Request-ID: req-ac559d48-fe99-4795-b21a-7332fe54717f) Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:20:57.151757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:20:57.152995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 13 04:20:57.154142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cf3a6f48-1b56-42de-9bed-3784483a02d2 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] Returning 404 to user: Volume 43033a98-054b-499b-9887-890a3a4ca28f could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:20:57.158365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cf3a6f48-1b56-42de-9bed-3784483a02d2 tempest-VolumesGetTestJSON-240553955 tempest-VolumesGetTestJSON-240553955-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f" status: 404 len: 109 microversion: 2.1 time: 0.092772 Oct 13 04:20:57.160117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 6/14] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:20:57 2022] GET /compute/v2.1/os-volumes/43033a98-054b-499b-9887-890a3a4ca28f => generated 109 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:20:57.437377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-76cde4df-a28b-4c2c-ba9e-cce21c659fbe tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a7b80>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:57.495996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-76cde4df-a28b-4c2c-ba9e-cce21c659fbe tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 200 len: 286 microversion: 2.1 time: 0.064240 Oct 13 04:20:57.501710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 9/15] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:20:57 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 286 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:58.508950 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-841c3b41-529e-4153-b4b7-7ee68ccd2011 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f0795840040>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:58.567959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-841c3b41-529e-4153-b4b7-7ee68ccd2011 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 200 len: 286 microversion: 2.1 time: 0.062884 Oct 13 04:20:58.573011 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 10/16] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:20:58 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 286 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:20:58.773811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c2fffa5a-3346-4522-b635-efdd72295976 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: .version_select at 0x7f07958a4e50>, body: {"keypair": {"name": "tempest-keypair-1115896741", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJU2BqOhC1R568CaQqQIaGcK1gvXjfuRHUhbs/D4P41RQ4W/GKvZy3CsD50RJPM3mBZtLp2HKXcUPXkc1mK9uDGKMV18aWRDoi9Xu62V8wxU2Os4KUhi0xanSkCPT4gmug=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:20:58.919687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-c2fffa5a-3346-4522-b635-efdd72295976 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:20:58.955443 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-c2fffa5a-3346-4522-b635-efdd72295976 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:20:59.071961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c2fffa5a-3346-4522-b635-efdd72295976 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.652895 Oct 13 04:20:59.080582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 7/17] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:20:58 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:59.582874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c34d42c4-951b-47c6-ad4b-ff12cdfcfe0c tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a7c10>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:59.645559 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c34d42c4-951b-47c6-ad4b-ff12cdfcfe0c tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 200 len: 287 microversion: 2.1 time: 0.067545 Oct 13 04:20:59.653504 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 11/18] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:20:59 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 287 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:20:59.663579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d2805d4f-b9f6-40d5-9513-1bbb43b0857e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f079577df70>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:59.828908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d2805d4f-b9f6-40d5-9513-1bbb43b0857e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "DELETE /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 202 len: 0 microversion: 2.1 time: 0.171251 Oct 13 04:20:59.829677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 8/19] 10.222.0.27 () {58 vars in 1254 bytes} [Thu Oct 13 04:20:59 2022] DELETE /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 0 bytes in 176 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:20:59.839413 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4ae95a3f-85fb-41db-aa76-0e2a155ee596 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a7550>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:20:59.902259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4ae95a3f-85fb-41db-aa76-0e2a155ee596 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 200 len: 286 microversion: 2.1 time: 0.067969 Oct 13 04:20:59.908334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 12/20] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:20:59 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 286 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:00.923912 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6c65dabe-0046-491a-a56b-190863110484 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07957b3670>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:00.986943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-6c65dabe-0046-491a-a56b-190863110484 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] HTTP exception thrown: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6c65dabe-0046-491a-a56b-190863110484 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Request method failure captured: Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c HTTP/1.1 Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IijpqvXk0iwKxx2H8jVWPth3FxXJPswhoQDkCQ56UPXe3mIo_J1ThMjYrk9cSILu8ALXW0cMpCHG_6hpZQgKU8eBHeIatVgYcdS0kEJtgKIVnxJQBKFDJmMeNxhR56nfCk5DapFhTCYxeC6_MTPCqHKe1Tywm5dxqx52lPKjQ8j4o Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:21:00.989232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f9a0de5ed456439092b335e5239a2df8", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 874fc04c47164bafb9ed3ab86f248dca Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '22fa4d17-0049-493d-afb4-4a463f4f8c7c'} Oct 13 04:21:00.990585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. (HTTP 404) (Request-ID: req-69c91135-1628-4c1f-a82f-ee2ca7400bb1) Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.991389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:00.992460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 13 04:21:00.993643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6c65dabe-0046-491a-a56b-190863110484 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Returning 404 to user: Snapshot 22fa4d17-0049-493d-afb4-4a463f4f8c7c could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6c65dabe-0046-491a-a56b-190863110484 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c" status: 404 len: 111 microversion: 2.1 time: 0.079075 Oct 13 04:21:00.994872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 9/21] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:21:00 2022] GET /compute/v2.1/os-snapshots/22fa4d17-0049-493d-afb4-4a463f4f8c7c => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:21:01.001606 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ea7745c8-250d-4ae7-b14e-48d21b362436 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f0795840820>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:01.219588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ea7745c8-250d-4ae7-b14e-48d21b362436 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 202 len: 0 microversion: 2.1 time: 0.223314 Oct 13 04:21:01.220511 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 13/22] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:21:00 2022] DELETE /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 0 bytes in 225 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:21:01.233878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a22cb62d-4254-4f09-8ef5-3f3d9bf9832c tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958a4a60>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:01.396835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a22cb62d-4254-4f09-8ef5-3f3d9bf9832c tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 200 len: 344 microversion: 2.1 time: 0.172065 Oct 13 04:21:01.398068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 10/23] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:01 2022] GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:02.419003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-307adba3-d0d4-4532-a53c-dea684a79d55 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f0795840af0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:02.496192 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-307adba3-d0d4-4532-a53c-dea684a79d55 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] HTTP exception thrown: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-307adba3-d0d4-4532-a53c-dea684a79d55 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Request method failure captured: Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 HTTP/1.1 Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IijpqvXk0iwKxx2H8jVWPth3FxXJPswhoQDkCQ56UPXe3mIo_J1ThMjYrk9cSILu8ALXW0cMpCHG_6hpZQgKU8eBHeIatVgYcdS0kEJtgKIVnxJQBKFDJmMeNxhR56nfCk5DapFhTCYxeC6_MTPCqHKe1Tywm5dxqx52lPKjQ8j4o Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:21:02.503326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f9a0de5ed456439092b335e5239a2df8", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 874fc04c47164bafb9ed3ab86f248dca Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '0f05c3aa-7c97-4c61-8611-fadb9f7c90e9'} Oct 13 04:21:02.504869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. (HTTP 404) (Request-ID: req-b7d45c14-fc7d-4e53-9072-f8b6de5e4005) Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.505703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:02.506995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 13 04:21:02.508282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-307adba3-d0d4-4532-a53c-dea684a79d55 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Returning 404 to user: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:02.509425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-307adba3-d0d4-4532-a53c-dea684a79d55 tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 404 len: 109 microversion: 2.1 time: 0.096868 Oct 13 04:21:02.515040 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 14/24] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:02 2022] GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 109 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:21:02.523054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-90a71427-c8ba-49ed-bd56-0457585d8fae tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07957b3310>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:02.612735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-90a71427-c8ba-49ed-bd56-0457585d8fae tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] HTTP exception thrown: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-90a71427-c8ba-49ed-bd56-0457585d8fae tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Request method failure captured: Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 HTTP/1.1 Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IijpqvXk0iwKxx2H8jVWPth3FxXJPswhoQDkCQ56UPXe3mIo_J1ThMjYrk9cSILu8ALXW0cMpCHG_6hpZQgKU8eBHeIatVgYcdS0kEJtgKIVnxJQBKFDJmMeNxhR56nfCk5DapFhTCYxeC6_MTPCqHKe1Tywm5dxqx52lPKjQ8j4o Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:21:02.615181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f9a0de5ed456439092b335e5239a2df8", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 874fc04c47164bafb9ed3ab86f248dca Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '0f05c3aa-7c97-4c61-8611-fadb9f7c90e9'} Oct 13 04:21:02.616670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. (HTTP 404) (Request-ID: req-d99c4602-17f2-472b-aecb-b8fd3c6962d6) Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.617581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 13 04:21:02.619026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 13 04:21:02.620656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-90a71427-c8ba-49ed-bd56-0457585d8fae tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Returning 404 to user: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-90a71427-c8ba-49ed-bd56-0457585d8fae tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 404 len: 109 microversion: 2.1 time: 0.100611 Oct 13 04:21:02.622862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 11/25] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:21:02 2022] DELETE /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 109 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:21:02.627444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b0b03848-3afb-4a3f-84fe-9b48f2e3507e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Calling method '.version_select at 0x7f07958401f0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:02.694603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-b0b03848-3afb-4a3f-84fe-9b48f2e3507e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] HTTP exception thrown: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b0b03848-3afb-4a3f-84fe-9b48f2e3507e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Request method failure captured: Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 HTTP/1.1 Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IijpqvXk0iwKxx2H8jVWPth3FxXJPswhoQDkCQ56UPXe3mIo_J1ThMjYrk9cSILu8ALXW0cMpCHG_6hpZQgKU8eBHeIatVgYcdS0kEJtgKIVnxJQBKFDJmMeNxhR56nfCk5DapFhTCYxeC6_MTPCqHKe1Tywm5dxqx52lPKjQ8j4o Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:21:02.696893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f9a0de5ed456439092b335e5239a2df8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f9a0de5ed456439092b335e5239a2df8", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: f9a0de5ed456439092b335e5239a2df8 Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-417560899 Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 874fc04c47164bafb9ed3ab86f248dca Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesSnapshotsTestJSON-417560899-project Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '0f05c3aa-7c97-4c61-8611-fadb9f7c90e9'} Oct 13 04:21:02.698260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. (HTTP 404) (Request-ID: req-9671b605-e792-4f90-b1ec-d1d4490427bb) Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.699029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:02.700106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 13 04:21:02.701167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b0b03848-3afb-4a3f-84fe-9b48f2e3507e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] Returning 404 to user: Volume 0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b0b03848-3afb-4a3f-84fe-9b48f2e3507e tempest-VolumesSnapshotsTestJSON-417560899 tempest-VolumesSnapshotsTestJSON-417560899-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9" status: 404 len: 109 microversion: 2.1 time: 0.077005 Oct 13 04:21:02.702729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 15/26] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:02 2022] GET /compute/v2.1/os-volumes/0f05c3aa-7c97-4c61-8611-fadb9f7c90e9 => generated 109 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:21:03.729013 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ef19157f-f17e-4019-b25f-aa52ebab2913 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Action: 'create', calling method: .version_select at 0x7f07957471f0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1577951638", "metadata": {"Type": "work"}}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ef19157f-f17e-4019-b25f-aa52ebab2913 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Request method failure captured: Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 122 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Iu8BcaxuAjezyswrQBYJUrpoEitGVfIGLtJygSLgDtXOStDVGeKVVUjMZpidX1VytHoPTaq3u_agNQ5zOAEi1TfmyeregGcLZ--ct9m2p7Rl1wDcemhXiyLDiEz5iMr-ci5iXLCkYUYPPV2sWiqPs1t5GeO4C2lUXD1K1gbFjVTy4 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:21:03.743477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f63ab8de0cd04a4e9e46b4d9292faae6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 450b79e9ba6e455aa07bcdd013b52d1f Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1577951638", "metadata": {"Type": "work"}}} Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: Oct 13 04:21:03.745083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1577951638', 'metadata': {'Type': 'work'}}}} Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise error Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi '#$%' Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:03.746090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 13 04:21:03.747629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 13 04:21:03.749829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 13 04:21:03.749829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.749829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ef19157f-f17e-4019-b25f-aa52ebab2913 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:03.749829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ef19157f-f17e-4019-b25f-aa52ebab2913 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.600296 Oct 13 04:21:03.755640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fb8fd201-7a4b-4d93-ac35-cfb369b41a88 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Action: 'create', calling method: .version_select at 0x7f07961ff9d0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1368582065", "metadata": {"Type": "work"}}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fb8fd201-7a4b-4d93-ac35-cfb369b41a88 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Request method failure captured: Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 120 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5Iu8BcaxuAjezyswrQBYJUrpoEitGVfIGLtJygSLgDtXOStDVGeKVVUjMZpidX1VytHoPTaq3u_agNQ5zOAEi1TfmyeregGcLZ--ct9m2p7Rl1wDcemhXiyLDiEz5iMr-ci5iXLCkYUYPPV2sWiqPs1t5GeO4C2lUXD1K1gbFjVTy4 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:21:03.761472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f63ab8de0cd04a4e9e46b4d9292faae6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 450b79e9ba6e455aa07bcdd013b52d1f Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1368582065", "metadata": {"Type": "work"}}} Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: Oct 13 04:21:03.763006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1368582065', 'metadata': {'Type': 'work'}}}} Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise error Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi '0' Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:03.763799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 13 04:21:03.765465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 13 04:21:03.767116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 13 04:21:03.767116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.767116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fb8fd201-7a4b-4d93-ac35-cfb369b41a88 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:03.767486 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 12/27] 10.222.0.27 () {60 vars in 1156 bytes} [Thu Oct 13 04:21:03 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 603 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:21:03.767636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fb8fd201-7a4b-4d93-ac35-cfb369b41a88 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.013681 Oct 13 04:21:03.775231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c5554811-37aa-4607-aaa8-1515413f655b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Action: 'create', calling method: .version_select at 0x7f07958a48b0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-111880784", "metadata": {"Type": "work"}}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:03.778544 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 16/28] 10.222.0.27 () {60 vars in 1156 bytes} [Thu Oct 13 04:21:03 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 27 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c5554811-37aa-4607-aaa8-1515413f655b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Request method failure captured: Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 118 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Iu8BcaxuAjezyswrQBYJUrpoEitGVfIGLtJygSLgDtXOStDVGeKVVUjMZpidX1VytHoPTaq3u_agNQ5zOAEi1TfmyeregGcLZ--ct9m2p7Rl1wDcemhXiyLDiEz5iMr-ci5iXLCkYUYPPV2sWiqPs1t5GeO4C2lUXD1K1gbFjVTy4 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:21:03.794246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f63ab8de0cd04a4e9e46b4d9292faae6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 450b79e9ba6e455aa07bcdd013b52d1f Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-111880784", "metadata": {"Type": "work"}}} Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: Oct 13 04:21:03.796657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-111880784', 'metadata': {'Type': 'work'}}}} Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise error Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi '' Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:03.797428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 13 04:21:03.798742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 13 04:21:03.799854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 13 04:21:03.799854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.799854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c5554811-37aa-4607-aaa8-1515413f655b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:03.799854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c5554811-37aa-4607-aaa8-1515413f655b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.024849 Oct 13 04:21:03.799854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 13/29] 10.222.0.27 () {60 vars in 1156 bytes} [Thu Oct 13 04:21:03 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 26 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:21:03.806106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-33647c35-7ae2-4b49-9f1b-3ae967904c8c tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Calling method '.version_select at 0x7f07957cf550>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:03.875086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-33647c35-7ae2-4b49-9f1b-3ae967904c8c tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] HTTP exception thrown: Volume tempest-invalid-950070720 could not be found. Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-33647c35-7ae2-4b49-9f1b-3ae967904c8c tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Request method failure captured: Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-950070720 HTTP/1.1 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5Iu8BcaxuAjezyswrQBYJUrpoEitGVfIGLtJygSLgDtXOStDVGeKVVUjMZpidX1VytHoPTaq3u_agNQ5zOAEi1TfmyeregGcLZ--ct9m2p7Rl1wDcemhXiyLDiEz5iMr-ci5iXLCkYUYPPV2sWiqPs1t5GeO4C2lUXD1K1gbFjVTy4 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:21:03.880102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f63ab8de0cd04a4e9e46b4d9292faae6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 450b79e9ba6e455aa07bcdd013b52d1f Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Volume tempest-invalid-950070720 could not be found. Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'tempest-invalid-950070720'} Oct 13 04:21:03.881600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-950070720 could not be found. (HTTP 404) (Request-ID: req-96a9d388-0a00-43a0-94d5-933fa66d3564) Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.882838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 13 04:21:03.883924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-950070720 could not be found. Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:03.884971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-950070720 could not be found. Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-950070720 could not be found. Oct 13 04:21:03.887161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.891918 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-33647c35-7ae2-4b49-9f1b-3ae967904c8c tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Returning 404 to user: Volume tempest-invalid-950070720 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:03.893285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-33647c35-7ae2-4b49-9f1b-3ae967904c8c tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/tempest-invalid-950070720" status: 404 len: 98 microversion: 2.1 time: 0.091048 Oct 13 04:21:03.894500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 17/30] 10.222.0.27 () {58 vars in 1215 bytes} [Thu Oct 13 04:21:03 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-950070720 => generated 98 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 13 04:21:03.903485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4de18932-f149-4e96-b3ea-962851075c87 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003880 Oct 13 04:21:03.904075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 14/31] 10.222.0.27 () {58 vars in 1140 bytes} [Thu Oct 13 04:21:03 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 13 04:21:03.913553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-475784a9-c4d0-4c7a-95e2-27810183d25f tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003979 Oct 13 04:21:03.928857 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-67c342b0-daaf-4ef2-b67f-27f10fab224b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Calling method '.version_select at 0x7f0795747c10>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:03.930257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 18/32] 10.222.0.27 () {58 vars in 1137 bytes} [Thu Oct 13 04:21:03 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 13 04:21:03.976826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-67c342b0-daaf-4ef2-b67f-27f10fab224b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] HTTP exception thrown: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-67c342b0-daaf-4ef2-b67f-27f10fab224b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Request method failure captured: Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/os-volumes/b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 HTTP/1.1 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Iu8BcaxuAjezyswrQBYJUrpoEitGVfIGLtJygSLgDtXOStDVGeKVVUjMZpidX1VytHoPTaq3u_agNQ5zOAEi1TfmyeregGcLZ--ct9m2p7Rl1wDcemhXiyLDiEz5iMr-ci5iXLCkYUYPPV2sWiqPs1t5GeO4C2lUXD1K1gbFjVTy4 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:21:03.979214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f63ab8de0cd04a4e9e46b4d9292faae6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 450b79e9ba6e455aa07bcdd013b52d1f Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6'} Oct 13 04:21:03.980750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. (HTTP 404) (Request-ID: req-395e85ed-8684-4a8f-b802-e9d4bc520320) Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.981902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 13 04:21:03.984447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 13 04:21:03.986768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. Oct 13 04:21:03.990120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:21:03.993758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-67c342b0-daaf-4ef2-b67f-27f10fab224b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Returning 404 to user: Volume b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:03.995959 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-67c342b0-daaf-4ef2-b67f-27f10fab224b tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6" status: 404 len: 109 microversion: 2.1 time: 0.076244 Oct 13 04:21:03.996937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 15/33] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:21:03 2022] DELETE /compute/v2.1/os-volumes/b6719ee4-c83e-4cfa-97d8-15c1fbdd65e6 => generated 109 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:21:04.008115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c59b2cc3-679b-4320-aec4-21265a6118c2 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Calling method '.version_select at 0x7f07958405e0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:04.062825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-c59b2cc3-679b-4320-aec4-21265a6118c2 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] HTTP exception thrown: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c59b2cc3-679b-4320-aec4-21265a6118c2 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Request method failure captured: Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/os-volumes/518ca217-3122-4269-b89c-8a0a5cc0e572 HTTP/1.1 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5Iu8BcaxuAjezyswrQBYJUrpoEitGVfIGLtJygSLgDtXOStDVGeKVVUjMZpidX1VytHoPTaq3u_agNQ5zOAEi1TfmyeregGcLZ--ct9m2p7Rl1wDcemhXiyLDiEz5iMr-ci5iXLCkYUYPPV2sWiqPs1t5GeO4C2lUXD1K1gbFjVTy4 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:21:04.065334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f63ab8de0cd04a4e9e46b4d9292faae6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f63ab8de0cd04a4e9e46b4d9292faae6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: f63ab8de0cd04a4e9e46b4d9292faae6 Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesNegativeTest-1906459904 Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 450b79e9ba6e455aa07bcdd013b52d1f Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesNegativeTest-1906459904-project Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '518ca217-3122-4269-b89c-8a0a5cc0e572'} Oct 13 04:21:04.066823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. (HTTP 404) (Request-ID: req-7a232e35-a2fe-4057-a879-0a8ccb6c999b) Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:04.068855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 13 04:21:04.069937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 13 04:21:04.071028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:21:04.072128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c59b2cc3-679b-4320-aec4-21265a6118c2 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] Returning 404 to user: Volume 518ca217-3122-4269-b89c-8a0a5cc0e572 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:21:04.074095 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c59b2cc3-679b-4320-aec4-21265a6118c2 tempest-VolumesNegativeTest-1906459904 tempest-VolumesNegativeTest-1906459904-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/518ca217-3122-4269-b89c-8a0a5cc0e572" status: 404 len: 109 microversion: 2.1 time: 0.070097 Oct 13 04:21:04.074883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 19/34] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:04 2022] GET /compute/v2.1/os-volumes/518ca217-3122-4269-b89c-8a0a5cc0e572 => generated 109 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:21:08.049453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1470026384", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "70d105db-93f5-4592-bb80-f58af13e321b"}], "security_groups": [{"name": "tempest-securitygroup--2009823630"}], "key_name": "tempest-keypair-1115896741", "user_data": "}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:08.710885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] validate_networks() for [('70d105db-93f5-4592-bb80-f58af13e321b', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:21:08.997122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:09.125709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1808753909", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "networks": [{"uuid": "656e3a17-1f34-48c5-89e8-17cf546de72a"}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:09.353885 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] validate_networks() for [('656e3a17-1f34-48c5-89e8-17cf546de72a', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:21:11.321980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:21:11.323148 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:21:11.325246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:21:11.325697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:21:11.326110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:21:11.361890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:11.374752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:11.375259 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:11.376082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:11.376548 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:11.376969 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:11.507905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:11.521917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:11.572694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:21:11.575937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:21:11.755270 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c835f64d-1dd5-4684-8cca-ae0099ac25df tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.711397 Oct 13 04:21:11.763160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 16/35] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:21:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 3720 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:21:11.784917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-86fea466-11c9-43b1-b00e-480b038f7fcf tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0795682700>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:11.804255 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:21:11.805563 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:21:11.806120 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:21:11.806780 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:21:11.807874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:21:11.891266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-86fea466-11c9-43b1-b00e-480b038f7fcf tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/36c9d2e2-2452-4395-9928-52d9ad626b79" status: 200 len: 351 microversion: 2.1 time: 0.125128 Oct 13 04:21:11.891843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 17/36] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:11 2022] GET /compute/v2.1/os-volumes/36c9d2e2-2452-4395-9928-52d9ad626b79 => generated 351 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:11.922513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:11.925020 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:11.927690 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:11.930543 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:11.938543 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:11.940405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:12.101516 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Getting quotas for project 2413ab4744c345bc94f68b6510ca3b44. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:12.161865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Getting quotas for user 70ff2e3f5a0f4e0da74af8d0ea6b1506 and project 2413ab4744c345bc94f68b6510ca3b44. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:12.200384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:21:12.203628 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:21:12.370445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-77722fd2-3725-41e3-aa18-67a143c07308 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.752231 Oct 13 04:21:12.372133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 20/37] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:21:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 3754 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 13 04:21:12.500070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-199893062", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:12.759342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] validate_networks() for [('618621b6-9258-44fe-b6fb-530eb2b4c8f1', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:21:12.829559 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9be205da-412e-44c6-b3a5-981cd97430a9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'create', calling method: .version_select at 0x7f0794ecc5e0>, body: {"keypair": {"name": "tempest-keypair-2076485658", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAUz93xjgL1Kch5W6seQ+5cwKLOM4IE53hagNmfVO7Uf85YzLlR5CmmQcIZLJc1NDsHeKWXGdtE2PuIrqWQEjg+g79rJXnZttHWHJRobLhT1DWGBCT3Qq+N+o/kkQRpYxw=="}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:12.907264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-9be205da-412e-44c6-b3a5-981cd97430a9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for project 75387c76ea384e5c96a336751c613e6e. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:12.919650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-9be205da-412e-44c6-b3a5-981cd97430a9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for user 35a058c849104a9699ecfcd0c81d398e and project 75387c76ea384e5c96a336751c613e6e. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:12.987313 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9be205da-412e-44c6-b3a5-981cd97430a9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.611818 Oct 13 04:21:12.988941 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 21/38] 10.222.0.27 () {62 vars in 1200 bytes} [Thu Oct 13 04:21:12 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 616 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 13 04:21:13.561405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5e532fd6-cc79-4d49-939f-4ab28831e709 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: .version_select at 0x7f0794ee8670>, body: {"keypair": {"name": "tempest-keypair-603749871", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEHksV/nz982TyopxikDndD4jV0B58SsbqxUiaCctOfp8qfQ1SmJGQZtfcflL0D5JLgIcpfSWgx4gOqy/kVZ973AlHA9UvtSHiY6rNrbfc+NDGv2hMTZCMXZFIIHDYJHag=="}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:13.605413 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-5e532fd6-cc79-4d49-939f-4ab28831e709 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:13.625743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-5e532fd6-cc79-4d49-939f-4ab28831e709 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for user 1f6cf725908041ccaceb1ffa694ad161 and project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:13.673745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5e532fd6-cc79-4d49-939f-4ab28831e709 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.682394 Oct 13 04:21:13.674762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 22/39] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:21:12 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 684 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:13.688061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-00371c71-727b-435e-8ec3-e33061eca9f5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:13.689427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-00371c71-727b-435e-8ec3-e33061eca9f5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:13.890837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-00371c71-727b-435e-8ec3-e33061eca9f5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:13.923633 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-00371c71-727b-435e-8ec3-e33061eca9f5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1242 microversion: 2.1 time: 0.245468 Oct 13 04:21:13.924386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 23/40] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:13 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1242 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:13.929820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-dbc075b0-26a4-4f2d-a5c0-a3687c563ef4 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:13.943254 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-dbc075b0-26a4-4f2d-a5c0-a3687c563ef4 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:14.049247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:21:14.049800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:21:14.050136 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:21:14.050570 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:21:14.050994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:21:14.077920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:14.078595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:14.084815 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:14.086226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:14.088846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:14.089493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:14.118596 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dbc075b0-26a4-4f2d-a5c0-a3687c563ef4 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:14.126572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-dbc075b0-26a4-4f2d-a5c0-a3687c563ef4 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1215 microversion: 2.1 time: 0.200329 Oct 13 04:21:14.130969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 24/41] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:13 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1215 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:14.136169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4a0e592e-bcd4-480f-bda2-2bc30b2a91be tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794ee84c0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:14.193750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for project 0913f8eb06164012b366d62c553d261c. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:14.212700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for user da9c75191a3b4ef49438aae9e35fcf13 and project 0913f8eb06164012b366d62c553d261c. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:14.256074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:21:14.262851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:21:14.272874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4a0e592e-bcd4-480f-bda2-2bc30b2a91be tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/acd17358-fe3f-4fda-9f4c-b22fe6e7a63e" status: 200 len: 351 microversion: 2.1 time: 0.140294 Oct 13 04:21:14.273601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 25/42] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:14 2022] GET /compute/v2.1/os-volumes/acd17358-fe3f-4fda-9f4c-b22fe6e7a63e => generated 351 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:14.363893 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-45d3a8e2-009a-4742-82c3-e1e0fa3c7e59 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.461611 Oct 13 04:21:14.369452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 18/43] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:21:11 2022] POST /compute/v2.1/servers => generated 383 bytes in 2477 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:21:14.384321 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3f63597c-c71f-43cd-b6bb-5170f5f64390 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:14.385094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3f63597c-c71f-43cd-b6bb-5170f5f64390 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:14.648832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3f63597c-c71f-43cd-b6bb-5170f5f64390 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:14.660845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3f63597c-c71f-43cd-b6bb-5170f5f64390 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1221 microversion: 2.1 time: 0.279264 Oct 13 04:21:14.692757 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 26/44] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:14 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1221 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:14.950872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:14.951681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:15.002734 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1018177543", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:15.010946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:15.026526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:15.028309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:15.036476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:15.036968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:15.037424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:15.200457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] validate_networks() for [('618621b6-9258-44fe-b6fb-530eb2b4c8f1', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:21:15.587934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:15.637647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5d33915b-6a10-4568-b6a8-f0e1715b111b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1238 microversion: 2.1 time: 0.685546 Oct 13 04:21:15.638351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 19/45] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:14 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1238 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:15.689908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:15.692583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:15.722159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:15.723044 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:15.728383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:15.729147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:15.737022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:15.737472 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:16.068094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:16.078847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3a1729ab-192f-4546-8d72-0664338db8a6 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1267 microversion: 2.1 time: 0.417395 Oct 13 04:21:16.080218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 20/46] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:15 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1267 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:16.431883 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0ecff4c5-11c8-476a-8bb4-b08c46a6f6e7 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794dc9f70>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:16.593679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:21:16.594789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:21:16.595595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:21:16.595985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:21:16.596417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:21:16.614743 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0ecff4c5-11c8-476a-8bb4-b08c46a6f6e7 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/a0343965-ac0f-47b6-8dbb-e98a85bd182d" status: 200 len: 350 microversion: 2.1 time: 0.196112 Oct 13 04:21:16.615915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 21/47] 10.222.0.27 () {58 vars in 1245 bytes} [Thu Oct 13 04:21:16 2022] GET /compute/v2.1/os-volumes/a0343965-ac0f-47b6-8dbb-e98a85bd182d => generated 350 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:16.631163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b31db92c-1d94-4a0c-b40c-ff57674f52d3 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794def160>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:16.662174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:16.668588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:16.669049 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:16.672639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:16.673114 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:16.674802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:16.733615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for project 0913f8eb06164012b366d62c553d261c. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:16.745911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for user da9c75191a3b4ef49438aae9e35fcf13 and project 0913f8eb06164012b366d62c553d261c. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:16.776888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:21:16.781593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:21:16.823266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b31db92c-1d94-4a0c-b40c-ff57674f52d3 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.196864 Oct 13 04:21:16.834437 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 22/48] 10.222.0.27 () {58 vars in 1134 bytes} [Thu Oct 13 04:21:16 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:16.839494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:16.845086 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:16.866672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:16.867169 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:16.867604 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:16.868311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:16.868765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:16.869192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:16.955539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7903ccf9-bb33-4a43-8412-927046b32ced tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.963155 Oct 13 04:21:16.960381 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 27/49] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:21:14 2022] POST /compute/v2.1/servers => generated 383 bytes in 1968 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:21:16.984680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-29674ddd-56bc-45de-b6bc-4f905fa944e1 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794eaa1f0>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:17.173868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:17.187859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-29674ddd-56bc-45de-b6bc-4f905fa944e1 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.226240 Oct 13 04:21:17.203758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 28/50] 10.222.0.27 () {58 vars in 1149 bytes} [Thu Oct 13 04:21:16 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:17.209710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5082ccba-684b-462e-929c-b8bf26087113 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:17.221457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d28c004c-c2d6-48fd-99af-3ff2dbb8636a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1294 microversion: 2.1 time: 0.383446 Oct 13 04:21:17.222570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5082ccba-684b-462e-929c-b8bf26087113 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:17.250371 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 23/51] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:16 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1294 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:17.267013 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:17.272061 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:17.285100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:17.287031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:17.290097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:17.291172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:17.293118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:17.295014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:17.783525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:17.794387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b8a5dd5c-6819-4dca-89ec-f13394d639fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1273 microversion: 2.1 time: 0.542535 Oct 13 04:21:17.799202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 24/52] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:17 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1273 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:17.816152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a4c2b9e6-0936-463c-85f8-fbdca49f7506 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794d7b550>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.010180 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5082ccba-684b-462e-929c-b8bf26087113 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:18.010737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a4c2b9e6-0936-463c-85f8-fbdca49f7506 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.209827 Oct 13 04:21:18.010941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 25/53] 10.222.0.27 () {58 vars in 1134 bytes} [Thu Oct 13 04:21:17 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:18.022386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5082ccba-684b-462e-929c-b8bf26087113 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1222 microversion: 2.1 time: 0.815687 Oct 13 04:21:18.023169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 29/54] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:17 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1222 bytes in 819 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:18.025593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7a9ea603-cd02-4e2f-a83e-fe622d21d8e3 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794d96a60>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.247218 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.248503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:18.250236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7a9ea603-cd02-4e2f-a83e-fe622d21d8e3 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.229035 Oct 13 04:21:18.250858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 26/55] 10.222.0.27 () {58 vars in 1167 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:21:18.266188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:18.268228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-da9cfed5-d716-4f3d-b25d-59ff50bed4fb tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794d37310>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.272875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:18.274652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:18.279667 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:18.280787 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:18.282728 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:18.467323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-da9cfed5-d716-4f3d-b25d-59ff50bed4fb tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.202897 Oct 13 04:21:18.468729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 27/56] 10.222.0.27 () {58 vars in 1170 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:18.486396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d3ea59f5-3dbd-4c7a-859d-2ef39ea276a8 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f07957b3b80>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.727314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d3ea59f5-3dbd-4c7a-859d-2ef39ea276a8 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.248241 Oct 13 04:21:18.728385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 28/57] 10.222.0.27 () {58 vars in 1155 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:18.745204 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c64b7ded-410f-4e57-9ce1-504854332049 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794d7b8b0>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.865139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:18.895761 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-86322eb0-9152-46c0-a0bb-f7a29e96183e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1300 microversion: 2.1 time: 0.653551 Oct 13 04:21:18.897028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 30/58] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1300 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:18.928208 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.931456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:18.954023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c64b7ded-410f-4e57-9ce1-504854332049 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.212887 Oct 13 04:21:18.955502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 29/59] 10.222.0.27 () {58 vars in 1188 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:21:18.972968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:18.976490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7830f091-f94b-472f-b37c-1b359ccc1b83 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] Calling method '.version_select at 0x7f0794d37dc0>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:18.980382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:18.995671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:18.996703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:18.998962 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:18.999581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:19.322352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7830f091-f94b-472f-b37c-1b359ccc1b83 tempest-VolumesTestJSON-1481294192 tempest-VolumesTestJSON-1481294192-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.351978 Oct 13 04:21:19.327718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 30/60] 10.222.0.27 () {58 vars in 1155 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:19.351334 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:19.352358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:19.370944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:19.372464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:19.377440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:19.378785 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:19.381969 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:19.382921 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:19.393807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:19.425862 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-dc33ecdd-0a03-4fad-8be0-5cf34d534e90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1273 microversion: 2.1 time: 0.505795 Oct 13 04:21:19.439805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 31/61] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:18 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1273 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:19.638978 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:19.669000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b049030c-c2db-4cac-8b60-380cfbe5eaee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1274 microversion: 2.1 time: 0.335359 Oct 13 04:21:19.671435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 31/62] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:19 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1274 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:19.941142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:19.941880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:19.992585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:19.993175 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:19.993641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:19.996555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-49860048", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "73173841-4fa1-43e4-b55e-5df3b82b728b"}], "security_groups": [{"name": "tempest-securitygroup--1807891385"}], "key_name": "tempest-keypair-2076485658", "user_data": "}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:20.006265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:20.006913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:20.007374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:20.311473 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:20.328843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2d063cf3-5592-4dc0-aec7-9b0f55df4774 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1300 microversion: 2.1 time: 0.409389 Oct 13 04:21:20.343184 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 32/63] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:19 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1300 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:20.454876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:20.456456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:20.470091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:20.470610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:20.471059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:20.471779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:20.496725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:20.497411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.025s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:20.880125 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:20.892476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9c03b3b1-4706-4446-b6f4-38ed9646436d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1273 microversion: 2.1 time: 0.442527 Oct 13 04:21:20.893298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 33/64] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:20 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1273 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:20.906827 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:20.908608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:20.923263 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:20.925269 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:20.925972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:20.927565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:20.929390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:20.930070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:21.111568 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] validate_networks() for [('73173841-4fa1-43e4-b55e-5df3b82b728b', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:21:21.239147 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:21.249720 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-80649305-1e63-460e-9558-7750e9218934 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1280 microversion: 2.1 time: 0.354505 Oct 13 04:21:21.274463 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 34/65] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:20 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1280 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:21.354049 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:21.355037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:21.373950 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:21.377581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:21.378341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:21.379206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:21.381071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:21.381585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:21.511181 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:21.521563 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-59a97819-08c2-4ab0-885d-6cd207f97dae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1300 microversion: 2.1 time: 0.171810 Oct 13 04:21:21.526734 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 35/66] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:21 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1300 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:21.538550 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2118479270", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "80a6f545-b088-4d6c-b95e-6cedd0b72a59"}], "security_groups": [{"name": "tempest-securitygroup--1936140612"}], "key_name": "tempest-keypair-603749871", "user_data": "}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:22.894771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=52,nova_api:INSERT=7 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:23.315514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] validate_networks() for [('80a6f545-b088-4d6c-b95e-6cedd0b72a59', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:21:24.136091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:24.249597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=12 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:24.893998 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:21:24.895036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:21:24.895697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:21:24.896348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:21:24.897165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:21:24.923328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:24.923845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:24.924289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:24.930601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:24.931326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:24.933803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:24.995464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for project 75387c76ea384e5c96a336751c613e6e. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:25.014528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for user 35a058c849104a9699ecfcd0c81d398e and project 75387c76ea384e5c96a336751c613e6e. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:25.051682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:21:25.053890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:21:25.284151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8d1f61d5-b97e-4052-a6a0-67c78a790104 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 5.292958 Oct 13 04:21:25.293205 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 32/67] 10.222.0.27 () {62 vars in 1188 bytes} [Thu Oct 13 04:21:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 5303 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 13 04:21:25.300636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:25.311035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:25.326812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:25.329509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:25.330025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:25.351241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:25.351241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:25.351241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:26.708556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=4 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:26.724238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:26.742442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a87c0ac5-7bb2-42c1-8108-069ecbe4f176 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 1.446510 Oct 13 04:21:26.746389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 33/68] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:25 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 1452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:26.762367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:26.763620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:26.782852 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:26.784456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:26.785758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:26.787879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:26.790301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:26.790947 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:28.016746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=28 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:28.105289 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:21:28.105739 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:21:28.109229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:21:28.109654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:21:28.110070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:21:28.136906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:28.142116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:28.144253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:28.151483 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:28.152456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:28.155090 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:28.235294 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:21:28.250548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for user 1f6cf725908041ccaceb1ffa694ad161 and project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:21:28.286017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:28.295259 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2e764d09-0d5c-426f-92b1-fc48f3fb38bb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 1.546841 Oct 13 04:21:28.296199 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 34/69] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:26 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 1549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:28.320059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:21:28.328627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:28.330425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:21:28.333005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:28.361846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:28.363716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:28.364798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:28.366714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:28.367399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:28.368943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:28.417932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:28.472832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-16e1b3cb-1964-458b-a96f-4cae5e7608d1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 6.943739 Oct 13 04:21:28.475214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 36/70] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:21:21 2022] POST /compute/v2.1/servers => generated 402 bytes in 6948 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:21:28.485877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:28.487029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:28.497422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:28.498187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:28.498933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:28.499912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:28.500647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:28.501299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:29.094462 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:29.111997 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:29.133259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:29.136506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e78d6c69-5883-464c-ac8e-3318e8f62091 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1332 microversion: 2.20 time: 0.660007 Oct 13 04:21:29.155881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 37/71] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:28 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1332 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:29.173334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:29.177673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:29.193995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:29.195946 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:29.197009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:29.200054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:29.203433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:29.204304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:30.041832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:30.058616 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fc2caa5a-35c7-40b1-81fd-fc6f4f040dbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 1.761332 Oct 13 04:21:30.064254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 35/72] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:28 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 1767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:30.410555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b96502c-fa69-49c2-a14b-ab5867ad711d", "tag": "6ff44094-2f0f-48d3-9ca9-a6dc0324f741"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:30.468740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:30.471963 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:30.478403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:30.579255 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Creating event network-changed:6ff44094-2f0f-48d3-9ca9-a6dc0324f741 for instance 5b96502c-fa69-49c2-a14b-ab5867ad711d on hv-cinder-86099 Oct 13 04:21:30.579798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118673) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 13 04:21:30.580203 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5476baf2-6547-4fc1-9611-32b33e729b14 req-961fef32-31e1-48c8-a6c9-bf68dd1ea19b service nova] Acquired lock "compute-rpcapi-router" {{(pid=118673) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 13 04:21:30.581273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:30.581729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:30.582158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:30.582982 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:30.583362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:30.583773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:30.634651 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.compute.rpcapi [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 13 04:21:30.635576 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] Releasing lock "compute-rpcapi-router" {{(pid=118673) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 13 04:21:30.683364 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6d6c1ea2-d769-4d53-8fd5-bcd56276bd6b None None] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.615663 Oct 13 04:21:30.686010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 36/73] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:30.704260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:30.705185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:30.727736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:30.730885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:30.732096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:30.733154 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:30.737215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:30.737927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:30.932461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:30.957282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0070c734-cbe4-4366-80e5-312be93b7e5b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1289 microversion: 2.1 time: 0.263115 Oct 13 04:21:30.958193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 37/74] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:30 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1289 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:30.986381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:30.999613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:31.024699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:31.025466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:31.027715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:31.029439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:31.031183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:31.032768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:31.539534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:31.559763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ce7921a6-2a58-47c9-852c-7d441a008575 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 2.399642 Oct 13 04:21:31.577792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:31.583144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 38/75] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:29 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 2426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:31.586963 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a8f6bb06-c237-476c-858d-f47e5fba52b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.614897 Oct 13 04:21:31.587535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 38/76] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:30 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:31.601976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:31.610589 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:31.611336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:31.611640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:31.619698 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:31.620430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:31.623301 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:31.624709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:31.626771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:31.628297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:31.631235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:31.637423 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:31.638609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:31.644775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:31.646268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:31.647716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:32.197860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:32.209967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:32.211486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:32.212636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-02d8b307-6f6c-4889-91da-8a30678282a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1338 microversion: 2.20 time: 0.624856 Oct 13 04:21:32.226538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 39/77] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:31 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1338 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:32.238397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:32.239385 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:32.245955 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:32.250187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:32.250892 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:32.251852 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:32.252538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:32.253185 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:32.528045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:32.536487 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-614ad01c-4424-43d0-94cb-3c41e67d3b90 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1307 microversion: 2.1 time: 0.308809 Oct 13 04:21:32.537484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 40/78] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:32 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1307 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:32.596597 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:32.597330 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:32.611389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:32.612867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:32.613785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:32.616215 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:32.618602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:32.619366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:32.866307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:32.880481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c82f6831-8fe0-4a97-bbb2-67136ee7d60a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 1.283034 Oct 13 04:21:32.907009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 39/79] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:31 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 1303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:32.907009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:32.907009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:32.918500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:32.925208 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:32.925812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:32.926660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:32.927294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:32.927723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.302650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:33.312231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-da828bc3-f136-46fa-ab5b-4975b8d3c944 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.412315 Oct 13 04:21:33.321913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:33.330571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 40/80] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:32 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:33.335323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-159d9572-1381-4bf9-aa7a-e02225a46d08 req-f1a695e3-7922-4b43-8388-1a499a4c6ffe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62033c3e-e894-4560-95b8-34e25cb33e72", "tag": "bbeaed93-8e22-4d9f-a3ed-cb1f27426334"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:33.339695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3f3c8be2-03fa-4dc2-b733-907c207270ea tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.748592 Oct 13 04:21:33.353548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 41/81] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:32 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:33.362396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-159d9572-1381-4bf9-aa7a-e02225a46d08 req-f1a695e3-7922-4b43-8388-1a499a4c6ffe service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.363984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:33.364607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:33.364824 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-159d9572-1381-4bf9-aa7a-e02225a46d08 req-f1a695e3-7922-4b43-8388-1a499a4c6ffe service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.365405 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-159d9572-1381-4bf9-aa7a-e02225a46d08 req-f1a695e3-7922-4b43-8388-1a499a4c6ffe service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.371354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.372275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.372907 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.374263 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.374913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.375540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.377107 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-159d9572-1381-4bf9-aa7a-e02225a46d08 req-f1a695e3-7922-4b43-8388-1a499a4c6ffe service nova] Creating event network-changed:bbeaed93-8e22-4d9f-a3ed-cb1f27426334 for instance 62033c3e-e894-4560-95b8-34e25cb33e72 on hv-cinder-86099 Oct 13 04:21:33.385296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-159d9572-1381-4bf9-aa7a-e02225a46d08 req-f1a695e3-7922-4b43-8388-1a499a4c6ffe service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053407 Oct 13 04:21:33.389711 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 41/82] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:33.569729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:33.570385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:33.578391 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:33.578689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.579317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.579941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.580771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.581374 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.581937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.594453 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:33.596750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:33.598169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d49c80e1-d234-4488-bef8-81f7d67a3e12 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1338 microversion: 2.20 time: 0.243582 Oct 13 04:21:33.615067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 42/83] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:33 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1338 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:33.654973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:33.663300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-743842f7-63b5-490f-9740-faa2e4194788 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1295 microversion: 2.1 time: 0.097903 Oct 13 04:21:33.666351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 42/84] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:33 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1295 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:33.922037 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:33.922868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:33.932098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.932593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.933026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.936765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:33.937245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:33.937680 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:33.977673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "68b9a887-b69f-4e94-b26b-95490a16f5f9", "tag": "946596f5-0fd5-4db0-a720-f291f93741ef"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:34.008160 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.015482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.016063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.073705 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Creating event network-changed:946596f5-0fd5-4db0-a720-f291f93741ef for instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 on hv-cinder-86099 Oct 13 04:21:34.076594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118674) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 13 04:21:34.077814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-9fc77b75-3907-48b2-ab75-fdda13754970 req-83c99b34-d750-4981-9313-8e28169dd1c5 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118674) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 13 04:21:34.082361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.083104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.085595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.088322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.089067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.090671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.121196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.compute.rpcapi [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 13 04:21:34.122132 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] Releasing lock "compute-rpcapi-router" {{(pid=118674) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 13 04:21:34.147502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-89ab4680-2059-4262-8a8e-c0513a42eeeb None None] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.531378 Oct 13 04:21:34.149450 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 43/85] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:34.334951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:34.335856 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:34.359676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.360448 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.361050 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.362118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.362726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.363291 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.519853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:34.537047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a1227b4f-40b5-49de-924c-b335ba4737fc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.619680 Oct 13 04:21:34.555045 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 43/86] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:33 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:34.560167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:34.561317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:34.572687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.573172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.573765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.574959 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.580203 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.580910 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.626679 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:34.785079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:34.793924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9905ae6f-e4a7-4c6f-9b72-6be8825a96aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.464647 Oct 13 04:21:34.810585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 44/87] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:34 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:34.819206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:34.819206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:34.839553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.844403 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.844824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:34.846414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:34.846895 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:34.848728 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:35.036560 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=6 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:35.161049 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:35.184205 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-663eb84a-85aa-48dc-b3de-858c590426cc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.621779 Oct 13 04:21:35.202532 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 44/88] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:34 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:35.213330 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:35.213946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:35.227574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:35.228778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:35.230886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:35.231927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:35.236745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:35.238104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:35.467017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:35.479379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-08125687-89d2-43d6-8519-5624e41a5789 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1295 microversion: 2.1 time: 0.275597 Oct 13 04:21:35.480243 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 45/89] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:35 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1295 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:35.563095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:35.564117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:35.571356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:35.572113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:35.572976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:35.574109 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:35.574919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:35.576085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:35.800695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:35.812682 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:35.815415 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:35.817959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3d9e90f5-348d-4400-9c8b-8c68ad4307dd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 1.005569 Oct 13 04:21:35.818750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 45/90] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:34 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 1008 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:35.839092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:35.839952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:35.848900 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:35.849664 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:35.850817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:35.852366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:35.853492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:35.853926 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.268913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:36.276847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-42aceb04-5a3b-4079-aec7-393e9e321a62 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.718176 Oct 13 04:21:36.281039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 46/91] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:35 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 723 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:36.288242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-5e009f5f-7ff3-4c9e-bf89-011265247abe req-b277e345-c965-4df7-961c-adea0e6fe9ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a3a0563-b11d-4cf6-8e95-0ec0884f227b", "tag": "d7911061-419d-4200-84cb-0fd55dbfabb4"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:36.308115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5e009f5f-7ff3-4c9e-bf89-011265247abe req-b277e345-c965-4df7-961c-adea0e6fe9ae service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.315205 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5e009f5f-7ff3-4c9e-bf89-011265247abe req-b277e345-c965-4df7-961c-adea0e6fe9ae service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.315783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5e009f5f-7ff3-4c9e-bf89-011265247abe req-b277e345-c965-4df7-961c-adea0e6fe9ae service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.336339 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-5e009f5f-7ff3-4c9e-bf89-011265247abe req-b277e345-c965-4df7-961c-adea0e6fe9ae service nova] Creating event network-changed:d7911061-419d-4200-84cb-0fd55dbfabb4 for instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b on hv-cinder-86099 Oct 13 04:21:36.348363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-5e009f5f-7ff3-4c9e-bf89-011265247abe req-b277e345-c965-4df7-961c-adea0e6fe9ae service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065959 Oct 13 04:21:36.350250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 47/92] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:36.361917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:36.363156 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:36.374022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.374547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.374987 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.378478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.378478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.378478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.718138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:36.727423 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4c356608-1732-441e-8a6f-6a4204cb36c5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.375907 Oct 13 04:21:36.728005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 48/93] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:36 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:36.736142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:36.740957 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:36.758414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.765082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.765675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.766502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.766972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.767464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.779724 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:36.793423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:36.801094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1e367955-3b10-4c78-b946-22199b8ab203 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.980795 Oct 13 04:21:36.802199 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 46/94] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:35 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 982 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:36.843344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:36.843949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:36.865798 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.870782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.874937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:36.876949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:36.878188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:36.879324 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:37.008373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:37.020660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a7fcf7e6-67f1-47f6-be8d-1cca33a042a7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.289175 Oct 13 04:21:37.021582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 49/95] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:36 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:37.174621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:37.184129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:37.189804 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:37.192947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5693a1dd-a25b-4a76-978e-011dc4b8b63f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.353510 Oct 13 04:21:37.208026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 47/96] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:36 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:21:37.309796 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:37.310754 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:37.321266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:37.321807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:37.322270 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:37.322948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:37.335161 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:37.336063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:37.662540 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:37.677181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-00fe62bc-43c9-4c1f-ab9f-d874a7cc75b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.372238 Oct 13 04:21:37.678039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 50/97] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:37 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:37.754689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:37.755341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:37.765276 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:37.768156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:37.768647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:37.772238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:37.772656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:37.773097 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:37.829931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:37.830293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:37.841614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:37.841614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:37.841614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:37.845970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:37.846490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:37.846930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.119277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:38.128683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e1a0b955-8076-47fa-8ee0-250d42acbd0b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.304359 Oct 13 04:21:38.129663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 51/98] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:37 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:38.135363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:38.136075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:38.142935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.143609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.144144 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.145053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.145709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.146387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.235139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:38.243853 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a0936c14-5272-4f3f-b2d1-122598244ecc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.493236 Oct 13 04:21:38.264494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 48/99] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:37 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:38.280253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:38.280875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:38.287061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.287615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.288056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.289099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.289986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.290627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.304258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:38.537735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:38.549317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ea3bf538-da54-4bbd-b6c5-2e6b31e2d9ad tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.418103 Oct 13 04:21:38.550234 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 52/100] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:38 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:38.555663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-9fd50fad-22de-4d60-ba4f-c9729dccb30d req-359c33a0-56d5-469b-ab0f-bc18e4d4ff58 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:38.587048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9fd50fad-22de-4d60-ba4f-c9729dccb30d req-359c33a0-56d5-469b-ab0f-bc18e4d4ff58 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.591118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9fd50fad-22de-4d60-ba4f-c9729dccb30d req-359c33a0-56d5-469b-ab0f-bc18e4d4ff58 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.591461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9fd50fad-22de-4d60-ba4f-c9729dccb30d req-359c33a0-56d5-469b-ab0f-bc18e4d4ff58 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.620574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-9fd50fad-22de-4d60-ba4f-c9729dccb30d req-359c33a0-56d5-469b-ab0f-bc18e4d4ff58 service nova] Creating event network-changed:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:21:38.635882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-9fd50fad-22de-4d60-ba4f-c9729dccb30d req-359c33a0-56d5-469b-ab0f-bc18e4d4ff58 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079476 Oct 13 04:21:38.636901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 53/101] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:38.702882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:38.703596 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:38.717871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.718452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.718897 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:38.719680 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:38.720247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:38.720733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.099448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.110523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-051c2ec9-4b04-4fef-a2d2-7811dab5c975 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.413560 Oct 13 04:21:39.129449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 54/102] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:38 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:39.140046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-fdb1ecd1-c699-4e25-ba3b-979183fed2dd req-e01d6411-ff4b-48f6-b957-4d00e232baf0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c318cdef-d4ee-46aa-bc4e-91a49cfe1985", "tag": "bbd5fde3-e147-43a0-a4cf-beb965d2c6c7"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:39.164019 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-fdb1ecd1-c699-4e25-ba3b-979183fed2dd req-e01d6411-ff4b-48f6-b957-4d00e232baf0 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.164496 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-fdb1ecd1-c699-4e25-ba3b-979183fed2dd req-e01d6411-ff4b-48f6-b957-4d00e232baf0 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.164917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-fdb1ecd1-c699-4e25-ba3b-979183fed2dd req-e01d6411-ff4b-48f6-b957-4d00e232baf0 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.184266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-fdb1ecd1-c699-4e25-ba3b-979183fed2dd req-e01d6411-ff4b-48f6-b957-4d00e232baf0 service nova] Creating event network-changed:bbd5fde3-e147-43a0-a4cf-beb965d2c6c7 for instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 on hv-cinder-86099 Oct 13 04:21:39.202565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-fdb1ecd1-c699-4e25-ba3b-979183fed2dd req-e01d6411-ff4b-48f6-b957-4d00e232baf0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071720 Oct 13 04:21:39.207269 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 55/103] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:39.221664 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:39.235212 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:39.241391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.241873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.242334 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.243027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.243645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.244077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.345840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.365368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.371303 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.378057 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6515f9fb-dcd2-4424-a95e-aa99e9545eb6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 1.109056 Oct 13 04:21:39.379159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 49/104] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:38 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 1114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:39.386125 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:39.388357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:39.396872 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.397726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.398565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.400041 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.400939 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.401701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.565180 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.581058 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b4aeadc5-7247-4081-8ea4-8cc560b948ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.366145 Oct 13 04:21:39.582918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 56/105] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:39 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:39.590411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:39.592356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:39.604324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.605211 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.607108 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.608380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:39.609788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:39.610330 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:39.929621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.938780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6426c9df-c200-4c93-bb70-90cf36aa2bbe tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.354429 Oct 13 04:21:39.951732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 57/106] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:39 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:39.979141 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:39.989542 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-88326e15-2071-49d2-9a42-5d07c0d422bd tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.609117 Oct 13 04:21:40.000850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 50/107] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:39 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:40.138463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:40.139689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:40.161364 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.162610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.163214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.164394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.165073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.165574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.378394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:40.387178 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ecda213d-10c8-4ecc-b87d-80b5221b84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.256661 Oct 13 04:21:40.388596 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 58/108] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:40 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:40.412988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:40.413642 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:40.421485 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.422038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.422499 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.423181 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.423952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.424658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.611553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:40.612092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:40.620905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.624968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.625577 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.626335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.629075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.629685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.630508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:40.900306 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:40.912218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-95663f6f-38ac-4460-ba4c-4e3c63fdfdbf tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.307038 Oct 13 04:21:40.916160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 59/109] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:40 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:40.917246 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:40.929182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:40.934022 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:40.940363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-20f22b94-460a-4761-be92-82e5c436e87c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.535606 Oct 13 04:21:40.952418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 51/110] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:40 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:40.975369 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:40.978243 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:40.993972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:40.995675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:40.996890 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:40.999139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.001112 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.003417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.177294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-79dfe93f-7a0b-4b46-b8f2-20942d0e51df req-d7e781ac-feb9-4082-b86b-ed8676875222 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b96502c-fa69-49c2-a14b-ab5867ad711d", "name": "network-vif-plugged", "status": "completed", "tag": "6ff44094-2f0f-48d3-9ca9-a6dc0324f741"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:41.208883 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-79dfe93f-7a0b-4b46-b8f2-20942d0e51df req-d7e781ac-feb9-4082-b86b-ed8676875222 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.210503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-79dfe93f-7a0b-4b46-b8f2-20942d0e51df req-d7e781ac-feb9-4082-b86b-ed8676875222 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.211168 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-79dfe93f-7a0b-4b46-b8f2-20942d0e51df req-d7e781ac-feb9-4082-b86b-ed8676875222 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.227158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-79dfe93f-7a0b-4b46-b8f2-20942d0e51df req-d7e781ac-feb9-4082-b86b-ed8676875222 service nova] Creating event network-vif-plugged:6ff44094-2f0f-48d3-9ca9-a6dc0324f741 for instance 5b96502c-fa69-49c2-a14b-ab5867ad711d on hv-cinder-86099 Oct 13 04:21:41.234446 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-79dfe93f-7a0b-4b46-b8f2-20942d0e51df req-d7e781ac-feb9-4082-b86b-ed8676875222 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.316637 Oct 13 04:21:41.235265 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 60/111] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:41.247066 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:41.248096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:41.256518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.257215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.257848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.259013 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.260226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.260807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.320928 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:41.334375 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c50287f0-8d0e-4e10-bd6f-4eff3ba422da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.371140 Oct 13 04:21:41.335606 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 52/112] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:40 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:41.425295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:41.425295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:41.442101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.442629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.443054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.443749 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.444198 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.444608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.545447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:41.554975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-22577dee-e664-474e-8224-8b6f5d315f94 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.312271 Oct 13 04:21:41.556174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 61/113] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:41 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:41.667553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:41.676562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-09e7279d-90dd-4db0-a1a3-152e50cee24a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.260379 Oct 13 04:21:41.681172 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 53/114] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:41 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:41.943067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:41.943956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:41.951503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.952303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.952956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.954543 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.955209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.955851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.973876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:41.975316 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:41.986842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.987534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.988273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:41.989212 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:41.989695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:41.992448 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.254944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.263694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ccc7d6d4-19c2-4fc1-b135-aaeaae8cf730 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.325336 Oct 13 04:21:42.267151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 62/115] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:41 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:42.361355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:42.362098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:42.370626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:42.371318 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:42.371848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.373983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:42.374687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:42.375720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.558835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.564712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:42.570812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5ff057ea-6992-4e90-8de6-f3ee9eba0752 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.216097 Oct 13 04:21:42.571407 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 63/116] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:42 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:42.583217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:42.584299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:42.591520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:42.592118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:42.592546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.593246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:42.593723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:42.594135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.749800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.761579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dc0d75eb-fb0c-4711-a225-a1dd9afd0552 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.181465 Oct 13 04:21:42.763117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 64/117] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:42 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:42.770452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:42.771422 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:42.782337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:42.782737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:42.783231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.784186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:42.784755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:42.785574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:42.958681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.967831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-bb2b051c-3486-40d9-8be6-5aac2f25fb86 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.202851 Oct 13 04:21:42.968952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 65/118] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:42 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:42.972419 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.987374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.988926 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:42.990113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1c86ce2a-a97c-4aac-91d5-f56eac222b97 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 1.020761 Oct 13 04:21:43.006774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 54/119] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:41 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 1030 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:43.252816 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62033c3e-e894-4560-95b8-34e25cb33e72", "name": "network-vif-plugged", "status": "completed", "tag": "bbeaed93-8e22-4d9f-a3ed-cb1f27426334"}, {"server_uuid": "68b9a887-b69f-4e94-b26b-95490a16f5f9", "name": "network-vif-plugged", "status": "completed", "tag": "946596f5-0fd5-4db0-a720-f291f93741ef"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:43.267399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.268443 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.269078 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.286738 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] Creating event network-vif-plugged:bbeaed93-8e22-4d9f-a3ed-cb1f27426334 for instance 62033c3e-e894-4560-95b8-34e25cb33e72 on hv-cinder-86099 Oct 13 04:21:43.287478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] Creating event network-vif-plugged:946596f5-0fd5-4db0-a720-f291f93741ef for instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 on hv-cinder-86099 Oct 13 04:21:43.295755 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:43.296377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:43.300508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-972ef2a5-15d0-437e-9642-4987d190c6c8 req-f0c1990d-696d-49c2-b9c1-e05ffb8d1960 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.050750 Oct 13 04:21:43.302133 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 66/120] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:43 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:43.312167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.315300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.332442 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.334603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.336759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.338993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.599076 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:43.599783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:43.616143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.617261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.618135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.620408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.621205 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.623621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.668602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:43.681585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cd1ff298-7489-4eb8-9e84-a473106237f5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.390885 Oct 13 04:21:43.682173 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 55/121] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:43 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:43.786076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:43.786885 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:43.796277 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.797113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.798385 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.800654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.802133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.806299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:43.822060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:43.831398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d22fd2b2-7a59-4139-9f84-bc0e40b7bda2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.238382 Oct 13 04:21:43.832295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 67/122] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:43 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:43.991723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:43.992339 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:43.998783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:43.999277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:43.999706 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.000398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.001008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.001583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.107471 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=4 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:44.161904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:44.171579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e3a473e2-ad7a-4ca7-a31c-7af2adf53bb8 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.390279 Oct 13 04:21:44.172408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 56/123] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:43 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:44.179527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:44.180383 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:44.190702 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.191210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.191638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.192349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.196482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.196958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.391798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:44.401579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b3e35baa-f134-49ff-8cd2-930381ee72c0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.413579 Oct 13 04:21:44.402152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 68/124] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:43 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:44.535046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:44.544985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:44.564717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:44.565903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-83ae2306-e185-4e4c-ae92-5c58891496bc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.390540 Oct 13 04:21:44.566977 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 57/125] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:44 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:44.705694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:44.706483 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:44.712904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.713501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.714026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.714864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.715476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.716661 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.855730 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:44.858048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:44.864337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.865165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.865912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.867221 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:44.867679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:44.868101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:44.886326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:44.904467 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8ecfd80d-5ea3-4057-9086-cffb902c4668 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.204131 Oct 13 04:21:44.911319 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 69/126] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:44 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:45.063379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:45.073822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f9b4dabf-8cb9-409a-b8e8-adc83761d92b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.222818 Oct 13 04:21:45.074753 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 58/127] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:44 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:45.194671 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:45.196111 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:45.202722 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.203384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.203864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.204607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.205052 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.205461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.339357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a3a0563-b11d-4cf6-8e95-0ec0884f227b", "name": "network-vif-plugged", "status": "completed", "tag": "d7911061-419d-4200-84cb-0fd55dbfabb4"}, {"server_uuid": "c318cdef-d4ee-46aa-bc4e-91a49cfe1985", "name": "network-vif-plugged", "status": "completed", "tag": "bbd5fde3-e147-43a0-a4cf-beb965d2c6c7"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:45.357508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.358002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.358475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.362989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:45.371375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f94692be-0d38-4106-b738-458b9d80d041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1315 microversion: 2.1 time: 0.180455 Oct 13 04:21:45.372104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 70/128] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:45 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1315 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:45.381887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] Creating event network-vif-plugged:d7911061-419d-4200-84cb-0fd55dbfabb4 for instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b on hv-cinder-86099 Oct 13 04:21:45.382423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] Creating event network-vif-plugged:bbd5fde3-e147-43a0-a4cf-beb965d2c6c7 for instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 on hv-cinder-86099 Oct 13 04:21:45.390664 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-28e0cfb2-6c58-4469-acce-8f2b13dc01de req-13591104-9d33-47b2-a0ed-d9022a71becb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.056580 Oct 13 04:21:45.391440 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 59/129] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:45 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:21:45.426247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:45.428342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:45.437272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.437272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.437272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.437272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.437903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.437903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.597374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:45.598059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:45.610651 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:45.611843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.612344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.612783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.613514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.613972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.614422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.623074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7b9f37a8-a53f-4cac-b50f-d244e14eacfd tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.200552 Oct 13 04:21:45.625781 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 71/130] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:45 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:45.890902 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:45.901232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:45.904206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:45.905340 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-20fbe9cc-02ef-4748-815b-5464675dd89c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.314801 Oct 13 04:21:45.906055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 60/131] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:45 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:45.934085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:45.937241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:45.945432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.945924 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.946386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:45.947620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:45.948074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:45.948680 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.101457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:46.102388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:46.113235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.117078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.118013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.119024 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.120941 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.125863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.145445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:46.154504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4bc1c078-4f88-4ad1-92cf-fc5e276e433a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.227788 Oct 13 04:21:46.163368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 72/132] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:45 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:46.354922 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:46.367082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-29f3fb85-8a87-4615-ac8a-3aa55472b483 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.270040 Oct 13 04:21:46.367670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 61/133] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:46 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:46.394773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:46.398158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:46.404599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.404973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.405411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.406138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.411906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.411906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.550150 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:46.557780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3aa2f833-6e8f-4e93-af6d-e9a4e4cbc4b7 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1502 microversion: 2.1 time: 0.166873 Oct 13 04:21:46.558650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 73/134] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:46 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1502 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:46.650090 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:46.650901 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:46.658763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.659465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.660430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.661336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.662395 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.663419 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.861404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:46.874374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1c7f19fb-4107-43b0-8b5b-4ecb715f16b9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.228369 Oct 13 04:21:46.876434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 62/135] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:46 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:46.938007 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:46.938759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:46.948046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.948532 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.948962 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:46.950565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:46.950961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:46.951410 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.086112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:47.094393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:47.095840 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:47.096946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3bca6dec-eca2-4e46-b6fb-80ff7c536afb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.164770 Oct 13 04:21:47.097506 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 74/136] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:46 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:21:47.180492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:47.181129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:47.187841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:47.188325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:47.188758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.189436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:47.189919 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:47.190371 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.327130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:47.335190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-58d827e8-cb11-4717-a956-2e33339a0ed2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.158491 Oct 13 04:21:47.335935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 63/137] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:47 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:47.392267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:47.392872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:47.394799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:47.401249 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:47.401731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:47.402158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.402871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:47.403353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:47.403923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.578519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:47.587723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-73ba3e7d-7f2e-49f6-91dc-5a89b59338e1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.199502 Oct 13 04:21:47.588302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 75/138] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:47 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:47.906509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:47.907360 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:47.922009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:47.922985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:47.923423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.924265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:47.925339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:47.926115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:47.931581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:48.081925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-837ee9a2-03e3-4639-8293-c5b0e961b019 req-faa8d75a-9ef9-469d-9ecb-53846ddc9be5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "name": "network-vif-plugged", "status": "completed", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:48.127307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-837ee9a2-03e3-4639-8293-c5b0e961b019 req-faa8d75a-9ef9-469d-9ecb-53846ddc9be5 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.127791 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-837ee9a2-03e3-4639-8293-c5b0e961b019 req-faa8d75a-9ef9-469d-9ecb-53846ddc9be5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.128227 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-837ee9a2-03e3-4639-8293-c5b0e961b019 req-faa8d75a-9ef9-469d-9ecb-53846ddc9be5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.173264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-837ee9a2-03e3-4639-8293-c5b0e961b019 req-faa8d75a-9ef9-469d-9ecb-53846ddc9be5 service nova] Creating event network-vif-plugged:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:21:48.188587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-837ee9a2-03e3-4639-8293-c5b0e961b019 req-faa8d75a-9ef9-469d-9ecb-53846ddc9be5 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.111454 Oct 13 04:21:48.191710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 76/139] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:21:48.202350 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:48.203933 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:48.217239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.220280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:48.223334 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.224145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.224883 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.227287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.227726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.229578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d01b8cf0-858a-45df-a870-496f20ba8784 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1368 microversion: 2.1 time: 0.329675 Oct 13 04:21:48.230771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 64/140] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:47 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1368 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:48.238906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6623aac8-3169-46f4-9f59-45765992d5e6"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:48.242257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:48.251263 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.252504 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.253156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.254068 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.254745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.255379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.448494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:48.460453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:48.462498 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:48.464006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-15c65638-c435-4110-a763-17289f7d6e58 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.270940 Oct 13 04:21:48.464850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 77/141] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:48 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:48.470600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:48.471554 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:48.477715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.478393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.479050 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.479935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.480631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.482666 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.682157 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:48.690657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-91838558-1ab2-4b86-9563-efb228a73701 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.223937 Oct 13 04:21:48.691225 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 78/142] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:48 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:48.695833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:48.696558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:48.703232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.703715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.704147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.704841 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:48.705295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:48.705714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:48.872350 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:48.881418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ec5a3892-ae87-41cf-8ed9-f803f99a5d13 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.189210 Oct 13 04:21:48.881994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 79/143] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:48 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:49.256471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:49.257070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:49.263018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.263486 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.264106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.264783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.265223 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.265635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.413278 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:49.422564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-69aee224-2b0a-488c-a566-1df1e433b22e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1558 microversion: 2.1 time: 0.169884 Oct 13 04:21:49.423142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 80/144] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:49 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:49.446260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:49.447030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:49.453595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.454182 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.456252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.456941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.457386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.457801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.579505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d429ab33-b0b1-4dcf-b259-9ace937d7cef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.137813 Oct 13 04:21:49.580295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 81/145] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:21:49 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-interface => generated 284 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:49.588309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:49.588309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:49.594364 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.594719 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.595186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.595809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.596251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.596665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.600879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:49.756888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:49.767401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:49.769305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:49.770914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f3aaa04c-1ad5-4f3c-baec-98299ab3ac81 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.189708 Oct 13 04:21:49.786240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 82/146] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:49 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:49.792610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:49.794051 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:49.802366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.802853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.803779 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.804925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.805392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.806120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.960535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:49.969953 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0f26790c-6452-4e6b-bfaa-3c75975ec9aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.181716 Oct 13 04:21:49.970879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 83/147] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:49 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:49.975656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:49.976351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:49.983584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.984215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.984881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:49.985802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:49.986600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:49.987023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:50.126250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:50.131593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c2533238-8100-4d9a-b10b-f868898b7af3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.159551 Oct 13 04:21:50.134243 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 84/148] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:49 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:50.799934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:50.799934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:50.808006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:50.808509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:50.808923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:50.809888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:50.810559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:50.811302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:50.996808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:51.007881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:51.009688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:51.011333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d55a42b8-ec81-4714-a5df-11a7f9e937ad tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.217979 Oct 13 04:21:51.011836 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 85/149] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:50 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:21:51.021803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:51.022452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:51.029399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:51.030134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:51.030660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:51.031636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:51.032277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:51.032707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:51.175834 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:51.183794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4bb4fa8a-84d8-4562-a783-ca2d1dd1216f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1322 microversion: 2.1 time: 0.170421 Oct 13 04:21:51.184530 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 86/150] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:51 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1322 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:51.188928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-ab4946d2-4308-48a0-a9c9-570837c0fd63 req-9054aa63-4099-424c-b13f-38aff4fa30d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62033c3e-e894-4560-95b8-34e25cb33e72", "tag": "bbeaed93-8e22-4d9f-a3ed-cb1f27426334"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:51.199536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-ab4946d2-4308-48a0-a9c9-570837c0fd63 req-9054aa63-4099-424c-b13f-38aff4fa30d8 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:51.200579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-ab4946d2-4308-48a0-a9c9-570837c0fd63 req-9054aa63-4099-424c-b13f-38aff4fa30d8 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:51.201135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-ab4946d2-4308-48a0-a9c9-570837c0fd63 req-9054aa63-4099-424c-b13f-38aff4fa30d8 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:51.214246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-ab4946d2-4308-48a0-a9c9-570837c0fd63 req-9054aa63-4099-424c-b13f-38aff4fa30d8 service nova] Creating event network-changed:bbeaed93-8e22-4d9f-a3ed-cb1f27426334 for instance 62033c3e-e894-4560-95b8-34e25cb33e72 on hv-cinder-86099 Oct 13 04:21:51.219563 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-ab4946d2-4308-48a0-a9c9-570837c0fd63 req-9054aa63-4099-424c-b13f-38aff4fa30d8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033841 Oct 13 04:21:51.220391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 87/151] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:51.225740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:51.226514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:51.234453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:51.235265 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:51.235697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:51.236499 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:51.237031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:51.237377 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:51.441525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:51.451599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-36e04be5-0e5a-4a3f-8029-1a438f3f8caf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.230038 Oct 13 04:21:51.453043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 88/152] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:51 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:52.045702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:52.047006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:52.062193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.062901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.063539 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.064347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.068279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.069024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.311371 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:52.337319 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:52.357882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:52.370712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4c13e8e2-c22e-4865-8819-ad1bf5babcab tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.333188 Oct 13 04:21:52.388744 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 89/153] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:52 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:52.404952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:52.406113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:52.415219 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.416063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.416710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.417622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.419162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.420365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.429991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a8032abb-7e0c-43ec-b2e8-880caddda9fc tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "POST /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.196793 Oct 13 04:21:52.431113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 65/154] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:21:48 2022] POST /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d/os-volume_attachments => generated 194 bytes in 4199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:52.478579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:52.479430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:52.491670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.492389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.492987 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.493856 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.494478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.495328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.728704 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:52.740145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-682c5b17-60c0-43bb-a7f7-2070a7f827aa tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1517 microversion: 2.1 time: 0.347959 Oct 13 04:21:52.741554 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 90/155] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:52 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1517 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:52.771604 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:52.782116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:52.792046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.794096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.795086 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.796015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:52.797409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:52.798043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:52.832677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'flavor' on Instance uuid 9a3a0563-b11d-4cf6-8e95-0ec0884f227b {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:21:52.894417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:52.906105 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e9ff4a47-a066-4163-bd0e-ca1836e76197 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1363 microversion: 2.1 time: 0.433767 Oct 13 04:21:52.911882 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 66/156] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:52 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1363 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:53.039274 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-729e6223-3e05-4a31-b038-af5ce87b4d31 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b/action" status: 200 len: 32 microversion: 2.1 time: 0.275481 Oct 13 04:21:53.040501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 91/157] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:21:52 2022] POST /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b/action => generated 32 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:21:53.052516 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:53.053523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:53.075408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:53.075859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:53.076541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:53.079655 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:53.080112 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:53.080884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:53.238143 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:53.247368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e083433b-fa75-4ffe-a1fd-12b1c4952412 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.200527 Oct 13 04:21:53.263054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 67/158] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:53 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:53.402501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:53.403817 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:53.410913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:53.411755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:53.412583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:53.413623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:53.414431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:53.415068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:53.576281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:53.589955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:53.591815 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:53.593132 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5283df11-0a65-4db7-9e7e-2760f9d48006 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.195677 Oct 13 04:21:53.594174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 92/159] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:53 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:53.940493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:53.942409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:53.957099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:53.957632 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:53.958056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:53.958949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:53.959589 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:53.960729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.153051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:54.164172 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bfd05457-5ad5-455e-afca-57e149b7527e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1548 microversion: 2.1 time: 0.231374 Oct 13 04:21:54.165266 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 68/160] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:53 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1548 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:54.188087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:54.188917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:54.194839 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:54.195316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:54.195919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.196612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:54.197059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:54.197479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.272678 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:54.273632 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:54.283836 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2d8b35eb-f27b-461a-a444-1d45cdac3ba6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099711 Oct 13 04:21:54.285305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:54.285440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 93/161] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:21:54 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:54.285542 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:54.285634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.286329 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:54.286944 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:54.287362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.619844 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:54.621201 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:54.629458 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:54.630321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:54.630997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.631930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:54.632584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:54.634882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:54.782443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:54.794574 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f76fa598-98a1-494b-b7c2-561fb773da8a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.527359 Oct 13 04:21:54.795187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 69/162] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:54 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:21:54.947789 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:54.960499 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:54.964027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:54.966291 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-058b340c-2341-4b4a-b3d6-74e46ea0995c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1406 microversion: 2.20 time: 0.350123 Oct 13 04:21:54.978944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 94/163] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:54 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1406 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:21:55.838335 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:55.839821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:55.851702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-dd0cf506-7e7e-4cc9-89b2-dc4b572c3534 req-91a90fe6-11ec-4156-af25-475281e77133 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c318cdef-d4ee-46aa-bc4e-91a49cfe1985", "tag": "bbd5fde3-e147-43a0-a4cf-beb965d2c6c7"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:55.864845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:55.866156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:55.869228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-dd0cf506-7e7e-4cc9-89b2-dc4b572c3534 req-91a90fe6-11ec-4156-af25-475281e77133 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:55.869767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:55.874135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:55.874135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:55.874135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:55.874604 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-dd0cf506-7e7e-4cc9-89b2-dc4b572c3534 req-91a90fe6-11ec-4156-af25-475281e77133 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:55.875529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-dd0cf506-7e7e-4cc9-89b2-dc4b572c3534 req-91a90fe6-11ec-4156-af25-475281e77133 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:55.895158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-dd0cf506-7e7e-4cc9-89b2-dc4b572c3534 req-91a90fe6-11ec-4156-af25-475281e77133 service nova] Creating event network-changed:bbd5fde3-e147-43a0-a4cf-beb965d2c6c7 for instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 on hv-cinder-86099 Oct 13 04:21:55.901750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-dd0cf506-7e7e-4cc9-89b2-dc4b572c3534 req-91a90fe6-11ec-4156-af25-475281e77133 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055166 Oct 13 04:21:55.903273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 95/164] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:55.998912 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:55.999902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:56.018685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:56.019492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:56.021693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:56.025586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:56.026309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:56.027203 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:56.310026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:56.319092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2c87efc7-92ce-49dc-a6d5-0c0650f75b89 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.495181 Oct 13 04:21:56.319588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 70/165] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:55 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:56.376821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:56.390399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:56.392393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:56.394133 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a7921437-6d9a-450c-a474-dd108b7b0ea6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1598 microversion: 2.20 time: 0.402981 Oct 13 04:21:56.408626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 96/166] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:21:55 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1598 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:21:56.435078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:56.436155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:56.444177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:56.445004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:56.445799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:56.446760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:56.447494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:56.450293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:56.944494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e4173479-f37e-4d8e-a3e8-25bff2b3d187 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.520108 Oct 13 04:21:56.945089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 71/167] 10.222.0.27 () {60 vars in 1316 bytes} [Thu Oct 13 04:21:56 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-interface => generated 284 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:21:57.349776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:57.350439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:57.364547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:57.365427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:57.369917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:57.370698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:57.371156 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:57.371615 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:58.319119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:58.329537 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-69d8d534-12e7-4907-b321-a058c162e2ec tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.985601 Oct 13 04:21:58.330449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 97/168] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:57 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 987 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:21:58.895199 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-216bfc34-5c5b-4cec-97aa-3419ec22b222 req-60042012-e40b-4f95-a2f1-994581b5c3fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:21:58.900380 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=71 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:58.907460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-216bfc34-5c5b-4cec-97aa-3419ec22b222 req-60042012-e40b-4f95-a2f1-994581b5c3fe service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:58.907956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-216bfc34-5c5b-4cec-97aa-3419ec22b222 req-60042012-e40b-4f95-a2f1-994581b5c3fe service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:58.908584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-216bfc34-5c5b-4cec-97aa-3419ec22b222 req-60042012-e40b-4f95-a2f1-994581b5c3fe service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:58.923216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-216bfc34-5c5b-4cec-97aa-3419ec22b222 req-60042012-e40b-4f95-a2f1-994581b5c3fe service nova] Creating event network-changed:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:21:58.929238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-216bfc34-5c5b-4cec-97aa-3419ec22b222 req-60042012-e40b-4f95-a2f1-994581b5c3fe service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037816 Oct 13 04:21:58.930505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 72/169] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:21:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:21:59.352456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:21:59.353325 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:21:59.365792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:59.369345 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:59.369888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:59.371314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:21:59.373290 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:21:59.374574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:21:59.415533 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:21:59.580747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:21:59.591627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-31787f8e-58da-412e-bead-0c4c4ce38a01 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.242993 Oct 13 04:21:59.595185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 98/170] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:21:59 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:00.621130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:00.623099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:00.630195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:00.630947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:00.631857 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:00.632568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:00.633557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:00.633964 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:00.830787 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:00.839126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bec7de96-9a07-4515-8a98-ae1d1377903c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.222256 Oct 13 04:22:00.840277 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 73/171] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:00 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:01.863528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:01.864467 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:01.873117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:01.873698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:01.874195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:01.875324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:01.875720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:01.876224 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:02.024146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:02.032990 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dc9efc9d-e264-4642-8dae-89d0a73e210f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.173558 Oct 13 04:22:02.036115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 99/172] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:01 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:03.059467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:03.060203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:03.066557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:03.067040 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:03.067483 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:03.068170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:03.068619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:03.069036 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:03.217136 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:03.225462 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-196e061d-9cee-4f64-850a-99c1595f22bd tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.170463 Oct 13 04:22:03.226282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 74/173] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:03 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:04.251291 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:04.251933 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:04.258166 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:04.258684 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:04.259151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:04.259942 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:04.260380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:04.260837 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:04.437584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:04.447542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-81932b8f-91cd-4342-8399-c0660ec45336 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.199998 Oct 13 04:22:04.448265 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 100/174] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:04 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:05.483577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:05.484362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:05.493200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:05.493689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:05.494121 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:05.494882 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:05.495333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:05.495750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:05.638239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:05.638239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:05.643984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:05.644464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:05.644887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:05.645565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:05.646011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:05.646450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:05.653400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:05.661832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b80e64f6-1d30-47c7-95da-9849ad7bf0b7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.184361 Oct 13 04:22:05.662438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 75/175] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:05 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:05.677707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:22:05.769718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-17dc3306-53dc-4734-9050-fae73b64843a tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "DELETE /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 204 len: 0 microversion: 2.1 time: 0.136235 Oct 13 04:22:05.773868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 101/176] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:22:05 2022] DELETE /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:22:05.779935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:05.780438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:05.788690 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:05.789303 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:05.789836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:05.790628 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:05.791161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:05.791637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:05.937797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:05.946607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-99281ac5-5a29-44bc-90c1-8d3e72a933d1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1554 microversion: 2.1 time: 0.171871 Oct 13 04:22:05.947181 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 76/177] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:05 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1554 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:06.687062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:06.687673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:06.693587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:06.694067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:06.694546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:06.695234 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:06.695689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:06.696110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:06.830398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:06.837912 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9677ad74-c22e-4744-a68a-93fc62e7f873 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.155276 Oct 13 04:22:06.838507 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 102/178] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:06 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:06.970950 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:06.971513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:06.978493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:06.978997 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:06.979662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:06.980450 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:06.981315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:06.981782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:07.113768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:07.124205 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-91debe4d-31cf-4c86-be14-8ec786311bf0 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1554 microversion: 2.1 time: 0.157172 Oct 13 04:22:07.124782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 77/179] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:06 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1554 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:07.861599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:07.862333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:07.868284 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:07.868764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:07.869387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:07.870081 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:07.870566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:07.871092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:07.999333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:08.007024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-613fa0c3-434b-4356-8f8e-d44b0cbda5c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.149369 Oct 13 04:22:08.007845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 103/180] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:07 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:08.148215 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:08.148751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:08.156011 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:08.156582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:08.157034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:08.157707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:08.158149 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:08.158589 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:08.288507 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:08.297295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9dc79894-3549-46f9-9af7-76c725744188 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1554 microversion: 2.1 time: 0.152998 Oct 13 04:22:08.297868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 78/181] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:08 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1554 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:09.033673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:09.034312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:09.040840 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:09.041534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:09.042128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:09.043075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:09.043687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:09.044266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:09.211887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:09.221664 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-77224f49-2409-4d04-8454-e6f2df243ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.191735 Oct 13 04:22:09.222556 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 104/182] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:09 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:09.323268 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:09.323877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:09.331497 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:09.331978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:09.332573 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:09.333447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:09.334102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:09.334691 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:09.615707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:09.625781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ba72b645-5787-4f18-998d-c87a8ac5b041 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1554 microversion: 2.1 time: 0.308621 Oct 13 04:22:09.626401 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 79/183] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:09 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1554 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:10.093357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-c60dd76b-d7f9-4e47-8222-7a07144ea4e1 req-2194d24a-7803-4481-8d48-a22cefdec451 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b96502c-fa69-49c2-a14b-ab5867ad711d", "name": "network-vif-deleted", "tag": "6ff44094-2f0f-48d3-9ca9-a6dc0324f741"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:10.106496 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c60dd76b-d7f9-4e47-8222-7a07144ea4e1 req-2194d24a-7803-4481-8d48-a22cefdec451 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:10.107123 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c60dd76b-d7f9-4e47-8222-7a07144ea4e1 req-2194d24a-7803-4481-8d48-a22cefdec451 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:10.107625 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c60dd76b-d7f9-4e47-8222-7a07144ea4e1 req-2194d24a-7803-4481-8d48-a22cefdec451 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:10.118989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-c60dd76b-d7f9-4e47-8222-7a07144ea4e1 req-2194d24a-7803-4481-8d48-a22cefdec451 service nova] Creating event network-vif-deleted:6ff44094-2f0f-48d3-9ca9-a6dc0324f741 for instance 5b96502c-fa69-49c2-a14b-ab5867ad711d on hv-cinder-86099 Oct 13 04:22:10.124378 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-c60dd76b-d7f9-4e47-8222-7a07144ea4e1 req-2194d24a-7803-4481-8d48-a22cefdec451 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035424 Oct 13 04:22:10.124988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 105/184] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:22:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:22:10.244686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:10.245345 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:10.251396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:10.251876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:10.252304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:10.252983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:10.253435 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:10.253858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:10.272325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:10.391696 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:10.399659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a34ef190-ef20-4de4-a7c9-ad6bedb885e5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.158772 Oct 13 04:22:10.400314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 80/185] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:10 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:10.651057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:10.651745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:10.658821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:10.659296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:10.659720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:10.660398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:10.660838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:10.661254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:10.726831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:10.735230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f5641df2-db93-4f71-95d4-71365db0dcd5 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1512 microversion: 2.1 time: 0.088371 Oct 13 04:22:10.736260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 106/186] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:10 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1512 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:11.432197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:11.433123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:11.441493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:11.442069 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:11.442605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:11.443384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:11.444610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:11.445114 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:11.578417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:11.586036 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cd90a3f4-06f0-407b-bc0a-4a12cb59aa9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.159133 Oct 13 04:22:11.586640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 81/187] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:11 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:11.758590 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:11.759863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:11.766801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:11.767283 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:11.767710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:11.768393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:11.768839 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:11.769260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:11.833657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:11.841835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-81fe82e2-1cc0-4e9a-964d-304e5b925af1 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1512 microversion: 2.1 time: 0.087249 Oct 13 04:22:11.842447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 107/188] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:11 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1512 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:12.610579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:12.611111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:12.617439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:12.617918 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:12.618377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:12.619063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:12.619511 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:12.619932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:12.755003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:12.762780 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0e92ada8-9b42-4850-b2fb-9199a4d1aeed tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.157090 Oct 13 04:22:12.764214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 82/189] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:12 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:12.865087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:12.865643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:12.871570 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:12.872044 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:12.872508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:12.873191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:12.873630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:12.874043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:12.935214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:12.946414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5bd8a6c2-86f5-4dc5-b668-4970bd2b988f tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1346 microversion: 2.1 time: 0.085091 Oct 13 04:22:12.947473 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 108/190] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:12 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1346 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:13.792033 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:13.792998 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:13.801586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:13.802317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:13.802975 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:13.803676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:13.804125 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:13.804545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:13.965323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:13.970349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:13.970949 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:13.976397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a712afe5-28fe-4981-a3a7-d7960f55884c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.194088 Oct 13 04:22:13.976966 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 83/191] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:13 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:13.978639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:13.979073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:13.979631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:13.980685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:13.981406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:13.982051 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:14.020746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:14.021236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:14.021666 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:14.022108 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lazy-loading 'fault' on Instance uuid 5b96502c-fa69-49c2-a14b-ab5867ad711d {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:22:14.084442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '70ff2e3f5a0f4e0da74af8d0ea6b1506', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2413ab4744c345bc94f68b6510ca3b44', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:14.094111 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-92a788b7-cb1e-499e-aea4-89c1227c951b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 200 len: 1351 microversion: 2.1 time: 0.128370 Oct 13 04:22:14.095889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 109/192] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:13 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 1351 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:15.003055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:15.005667 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:15.017248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:15.017981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:15.018660 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:15.020311 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:15.027529 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:15.028288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:15.119377 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:15.125669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:15.136738 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:15.137233 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:15.137715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:15.138687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:15.139264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:15.139959 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:15.156822 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] HTTP exception thrown: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Request method failure captured: Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d HTTP/1.1 Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I0gsJhzAXPTxQg9faBzezYZNHg7xjiDGrMMgl9igBhB7jGKolBSFT2kpiV3W31TSJj6hb_LwNzny_3jXVbJmPlxLkAjl5mHtumsk-VXTpktxWaWVEvUbQo6VPQTi5N-DEL4h7Wur3KQgorfvMGJYIVKW8OgodlVB4Tj6Bo0EZ_-VI Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:22:15.161916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_2413ab4744c345bc94f68b6510ca3b44", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 70ff2e3f5a0f4e0da74af8d0ea6b1506 Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.163321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '5b96502c-fa69-49c2-a14b-ab5867ad711d'} Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:22:15.164110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.165468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.167403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Returning 404 to user: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:22:15.167403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-988083e7-c07a-4111-8e9b-abe90ba32a90 tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 404 len: 111 microversion: 2.1 time: 0.050152 Oct 13 04:22:15.167403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 110/193] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:15 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:22:15.190876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:15.199026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cf6b7667-a9a3-4def-892a-4293389105ac tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.201946 Oct 13 04:22:15.199707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 84/194] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:14 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:15.301597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:15.303884 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:15.310071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:15.310789 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:15.311227 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:15.311920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:15.312369 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:15.312784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:15.327444 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] HTTP exception thrown: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Request method failure captured: Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d/os-volume_attachments HTTP/1.1 Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I0gsJhzAXPTxQg9faBzezYZNHg7xjiDGrMMgl9igBhB7jGKolBSFT2kpiV3W31TSJj6hb_LwNzny_3jXVbJmPlxLkAjl5mHtumsk-VXTpktxWaWVEvUbQo6VPQTi5N-DEL4h7Wur3KQgorfvMGJYIVKW8OgodlVB4Tj6Bo0EZ_-VI Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:22:15.329118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_2413ab4744c345bc94f68b6510ca3b44", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 70ff2e3f5a0f4e0da74af8d0ea6b1506 Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.330806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'server_id': '5b96502c-fa69-49c2-a14b-ab5867ad711d'} Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:22:15.331767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:15.333091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:15.334445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Returning 404 to user: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:22:15.334445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5378a74c-b2cc-40ab-9427-1cb8332f459d tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.033032 Oct 13 04:22:15.334445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 111/195] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:22:15 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d/os-volume_attachments => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:22:16.224601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:16.225672 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:16.237397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:16.238198 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:16.238808 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:16.240006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:16.240674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:16.241186 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:16.383224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:16.392092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f094efaa-34fc-4a1b-a307-ae02a838109a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.171861 Oct 13 04:22:16.392671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 85/196] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:16 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:16.546266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:16.546856 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:16.553365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:16.553842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:16.554323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:16.555012 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:16.555485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:16.555900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:16.569961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] HTTP exception thrown: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Request method failure captured: Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d HTTP/1.1 Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I0gsJhzAXPTxQg9faBzezYZNHg7xjiDGrMMgl9igBhB7jGKolBSFT2kpiV3W31TSJj6hb_LwNzny_3jXVbJmPlxLkAjl5mHtumsk-VXTpktxWaWVEvUbQo6VPQTi5N-DEL4h7Wur3KQgorfvMGJYIVKW8OgodlVB4Tj6Bo0EZ_-VI Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:22:16.571525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_2413ab4744c345bc94f68b6510ca3b44", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 70ff2e3f5a0f4e0da74af8d0ea6b1506 Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.573286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '5b96502c-fa69-49c2-a14b-ab5867ad711d'} Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:22:16.574372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.576261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.578009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Returning 404 to user: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:22:16.578009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-75e842be-05b1-46bb-8c9b-a2c902ff6a6b tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "DELETE /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 404 len: 111 microversion: 2.1 time: 0.030897 Oct 13 04:22:16.578009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 112/197] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:22:16 2022] DELETE /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:22:16.583622 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:16.584304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] [instance: 5b96502c-fa69-49c2-a14b-ab5867ad711d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:16.590373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:16.590856 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:16.591306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:16.591991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:16.592475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:16.593008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:16.607408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] HTTP exception thrown: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Request method failure captured: Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d HTTP/1.1 Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I0gsJhzAXPTxQg9faBzezYZNHg7xjiDGrMMgl9igBhB7jGKolBSFT2kpiV3W31TSJj6hb_LwNzny_3jXVbJmPlxLkAjl5mHtumsk-VXTpktxWaWVEvUbQo6VPQTi5N-DEL4h7Wur3KQgorfvMGJYIVKW8OgodlVB4Tj6Bo0EZ_-VI Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:22:16.613032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/2413ab4744c345bc94f68b6510ca3b44"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_2413ab4744c345bc94f68b6510ca3b44", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 2413ab4744c345bc94f68b6510ca3b44 Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-DeleteServersTestJSON-2074044317 Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 70ff2e3f5a0f4e0da74af8d0ea6b1506 Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-DeleteServersTestJSON-2074044317-project Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.614451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '5b96502c-fa69-49c2-a14b-ab5867ad711d'} Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:22:16.615332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. Oct 13 04:22:16.616639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:22:16.619293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] Returning 404 to user: Instance 5b96502c-fa69-49c2-a14b-ab5867ad711d could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:22:16.620759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b5d94f35-fbfd-49a7-9379-07d7e4d397da tempest-DeleteServersTestJSON-2074044317 tempest-DeleteServersTestJSON-2074044317-project] 10.222.0.27 "GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d" status: 404 len: 111 microversion: 2.1 time: 0.040684 Oct 13 04:22:16.621630 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 86/198] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:16 2022] GET /compute/v2.1/servers/5b96502c-fa69-49c2-a14b-ab5867ad711d => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:22:17.418913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:17.421008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:17.432836 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:17.436518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:17.437224 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:17.438928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:17.441233 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:17.442809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:17.444112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:17.727046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:17.735521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e9ed0c59-1c70-4ca2-9f4d-5973ac3268ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.321516 Oct 13 04:22:17.736094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 113/199] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:17 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:18.759329 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:18.759939 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:18.769768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:18.772599 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:18.773056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:18.773778 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:18.774269 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:18.775624 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:19.053023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1406401423", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "networks": [{"uuid": "7358a795-2926-4538-a53c-de076264b109"}]}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:19.152960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] validate_networks() for [('7358a795-2926-4538-a53c-de076264b109', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:22:19.198565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:19.210970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b5c0151f-210c-4630-8d50-22e8de77a0cc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.455553 Oct 13 04:22:19.226237 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 87/200] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:18 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:19.641795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:22:19.642555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:22:19.643490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:22:19.644074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:22:19.644628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:22:19.653973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:19.654762 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:19.655196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:19.655927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:19.656380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:19.656798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:19.689039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Getting quotas for project e7d1e420eec84d1f8b78333b7d814f3c. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:22:19.694574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Getting quotas for user 9ee6eac2e6da4d83bd14324669dc7269 and project e7d1e420eec84d1f8b78333b7d814f3c. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:22:19.716056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:22:19.719489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:22:19.823085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fcce3ce6-56a8-4d47-bff5-bf7bea0e9414 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.770488 Oct 13 04:22:19.835020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 114/201] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:22:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 787 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 13 04:22:19.840197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8a8f0051-f3b7-440e-b92c-8587d04b1fab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:19.840817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8a8f0051-f3b7-440e-b92c-8587d04b1fab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:19.943655 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8a8f0051-f3b7-440e-b92c-8587d04b1fab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:19.952536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8a8f0051-f3b7-440e-b92c-8587d04b1fab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1214 microversion: 2.1 time: 0.117235 Oct 13 04:22:19.964103 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 88/202] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:19 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1214 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:20.238378 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:20.239549 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:20.251945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:20.252459 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:20.252900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:20.258183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:20.258908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:20.259511 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:20.525424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:20.537570 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fc267272-069f-4d45-bfd8-e71c0520017f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.303456 Oct 13 04:22:20.538517 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 115/203] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:20 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:20.976660 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:20.978022 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:20.992574 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:20.996974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:20.998688 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:20.999713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:21.000311 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:21.000874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:21.080069 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:21.089752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-26b91c89-c31e-46c1-9c3c-2906f2708207 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1266 microversion: 2.1 time: 0.117449 Oct 13 04:22:21.090348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 89/204] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:20 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1266 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:21.564927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:21.566341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:21.575035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:21.576204 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:21.576722 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:21.577568 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:21.578100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:21.578920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:21.623807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:21.853323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:21.865337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-caa1edfd-a45b-4dc1-9dde-030f8a9f440a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.305431 Oct 13 04:22:21.865923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 116/205] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:21 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:22.116783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:22.116783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:22.128396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:22.128881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:22.129310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:22.130001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:22.130581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:22.130993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:22.209164 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:22.218417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-94c708ac-7b8c-479b-8b07-69bd7e86ba57 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1272 microversion: 2.1 time: 0.108169 Oct 13 04:22:22.219189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 90/206] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:22 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1272 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:22.892394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:22.893010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:22.898943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:22.899431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:22.899868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:22.900557 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:22.901011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:22.901435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:23.068379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:23.077336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6aefedc6-d6c2-42dc-a038-b484d18e74d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.188721 Oct 13 04:22:23.077970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 117/207] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:22 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:23.242277 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:23.242905 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:23.250637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:23.251126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:23.251558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:23.252250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:23.252700 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:23.253118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:23.408970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:23.416658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1db71963-9d9d-4bc9-8266-bbf0bfef4170 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.178179 Oct 13 04:22:23.417227 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 91/208] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:23 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:24.101595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:24.102125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:24.109199 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:24.109672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:24.110102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:24.110799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:24.111252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:24.111667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:24.237033 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:24.244774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5a6d0409-4367-4ce4-b9a6-2a44cac69a7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.147068 Oct 13 04:22:24.245349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 118/209] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:24 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:24.438504 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:24.439310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:24.445804 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:24.446334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:24.446774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:24.447464 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:24.447915 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:24.448341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:24.588683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:24.597175 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ea8b871-7655-4daa-a30d-a7574011f60d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.162716 Oct 13 04:22:24.597748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 92/210] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:24 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:25.268723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:25.269361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:25.275382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:25.275863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:25.276365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:25.277693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:25.278380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:25.279219 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:25.413758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:25.423210 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dc722c94-b2e5-4889-92fe-6ecbcffbdbe3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.158305 Oct 13 04:22:25.423868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 119/211] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:25 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:25.620386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:25.620988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:25.627727 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:25.628203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:25.628629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:25.629307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:25.629749 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:25.630188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:25.838945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:25.847334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-727179a9-32a1-4fb8-8377-f2896af14040 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.230792 Oct 13 04:22:25.847911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 93/212] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:25 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:25.955420 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-84e61847-895c-4ce0-9da5-f0de6e900284 req-8633dfdd-6a38-48bd-a807-b4170ffd64ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1af8fd18-6186-4269-aa97-d5bc623c9b7f", "tag": "ad559ec1-ddc7-418a-ab7d-758dffdd11af"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:25.964656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-84e61847-895c-4ce0-9da5-f0de6e900284 req-8633dfdd-6a38-48bd-a807-b4170ffd64ef service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:25.965121 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-84e61847-895c-4ce0-9da5-f0de6e900284 req-8633dfdd-6a38-48bd-a807-b4170ffd64ef service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:25.965540 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-84e61847-895c-4ce0-9da5-f0de6e900284 req-8633dfdd-6a38-48bd-a807-b4170ffd64ef service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:25.977022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-84e61847-895c-4ce0-9da5-f0de6e900284 req-8633dfdd-6a38-48bd-a807-b4170ffd64ef service nova] Creating event network-changed:ad559ec1-ddc7-418a-ab7d-758dffdd11af for instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f on hv-cinder-86099 Oct 13 04:22:25.982760 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-84e61847-895c-4ce0-9da5-f0de6e900284 req-8633dfdd-6a38-48bd-a807-b4170ffd64ef service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031349 Oct 13 04:22:25.983340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 120/213] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:22:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:22:26.445880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:26.446509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:26.453132 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:26.453610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:26.454039 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:26.454744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:26.455195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:26.455615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:26.584812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:26.594178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f79da9c6-caeb-4c89-bae6-a51e245b8a5a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.151813 Oct 13 04:22:26.594928 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 94/214] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:26 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:26.872036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:26.873413 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:26.880403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:26.880937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:26.881336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:26.882055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:26.882560 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:26.883036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:27.039192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:27.052950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-40b8f6d9-f3ec-4008-8200-992674360db1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.185491 Oct 13 04:22:27.053544 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 121/215] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:26 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:27.619989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:27.620594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:27.627171 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:27.627654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:27.628083 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:27.628773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:27.629223 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:27.629647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:27.765006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:27.774183 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a6bff7c2-b01f-434d-8c76-b728e57867b5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.158430 Oct 13 04:22:27.774863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 95/216] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:27 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:28.079005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:28.079799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:28.086136 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:28.086640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:28.087065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:28.087750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:28.088189 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:28.088601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:28.221820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:28.229307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2b4fc8fe-2287-430f-90d3-a0d561a62888 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.156244 Oct 13 04:22:28.229868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 122/217] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:28 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:28.797396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:28.798117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:28.804608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:28.805088 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:28.805543 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:28.806249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:28.806701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:28.807122 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:28.939728 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:28.947382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-177696f1-fe8b-464d-88be-fb7f7094dd03 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.154517 Oct 13 04:22:28.947946 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 96/218] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:28 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:29.252731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:29.253335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:29.259214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:29.259693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:29.260124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:29.260805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:29.261253 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:29.261672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:29.391841 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:29.399439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a4644c85-5693-4e3c-8ca7-07b68f2170be tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.150543 Oct 13 04:22:29.400002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 123/219] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:29 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:29.970584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:29.971219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:29.973511 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:29.974139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=52 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:29.979689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:29.980167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:29.980595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:29.981539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:29.982679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:29.983121 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:30.108375 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:30.115951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-32a475fd-c5de-44a4-879c-4e8c53a141d8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.149739 Oct 13 04:22:30.116524 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 97/220] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:29 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:30.422318 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:30.422804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:30.424813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:30.425655 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=6 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:30.431297 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:30.431775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:30.432201 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:30.432891 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:30.433337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:30.433753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:30.564519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:30.572039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-064e62d1-945d-48cd-8cc9-5a62d06d2cba tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.154073 Oct 13 04:22:30.572637 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 124/221] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:30 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:31.138746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:31.139354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:31.145704 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:31.146186 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:31.146696 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:31.147379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:31.147829 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:31.148250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:31.283862 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:31.287312 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:31.292557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-afae85a8-4565-4f7b-90ed-febcf9d074e3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.157720 Oct 13 04:22:31.293131 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 98/222] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:31 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:31.597956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:31.598954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:31.610932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:31.610932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:31.610932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:31.610932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:31.611279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:31.611629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:31.747709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:31.755850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7efd9956-919f-48ea-9b1c-502984b39571 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.164203 Oct 13 04:22:31.756322 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 125/223] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:31 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:32.316360 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:32.316867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:32.323205 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:32.323681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:32.324106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:32.324777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:32.325250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:32.325711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:32.453964 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:32.461725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9d20277e-1183-4440-9a58-5c70b09ca5b8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.149462 Oct 13 04:22:32.462367 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 99/224] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:32 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:32.779082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:32.779819 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:32.785650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:32.786141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:32.786610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:32.787346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:32.787805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:32.788231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:32.918080 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:32.926411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ad13f0ed-c8a1-4f45-a39e-c51a15414ec9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.151167 Oct 13 04:22:32.926978 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 126/225] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:32 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:33.484844 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:33.485478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:33.491867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:33.492346 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:33.492778 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:33.493457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:33.493907 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:33.494347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:33.633398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:33.641752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-12e8af50-ec90-475e-a0b0-d75eb5385709 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.161060 Oct 13 04:22:33.642348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 100/226] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:33 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:33.949362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:33.949959 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:33.958321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:33.958806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:33.959238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:33.959930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:33.960373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:33.960785 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:34.102833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:34.111676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fea49e82-0333-40c0-923d-5c038a9d0a0b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.166098 Oct 13 04:22:34.112266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 127/227] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:33 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:34.665497 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:34.666280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:34.673194 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:34.673675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:34.674102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:34.674819 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:34.675267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:34.675686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:34.811484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:34.819138 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2784425e-4551-46b2-9b64-2092cf4da0e9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1523 microversion: 2.1 time: 0.157727 Oct 13 04:22:34.819700 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 101/228] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:34 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1523 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:35.134836 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:35.135679 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:35.142889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:35.143375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:35.143811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:35.144500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:35.144953 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:35.145401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:35.282675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:35.290307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-61f8a50e-b581-49c4-8ac5-47e3cb67afab tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.159737 Oct 13 04:22:35.290872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 128/229] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:35 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:35.842564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:35.843162 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:35.849722 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:35.850251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:35.850688 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:35.851369 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:35.851818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:35.852236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:35.987804 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:35.995469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-dfa289da-e518-48f1-9bfb-9f17ff816f38 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1503 microversion: 2.1 time: 0.157107 Oct 13 04:22:35.996030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 102/230] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:35 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:36.019394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:36.020395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:36.027644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:36.028314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:36.029502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:36.030737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:36.031202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:36.031632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:36.186029 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:36.194026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cf1525c5-36de-4543-bb30-ad12f29574c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1515 microversion: 2.1 time: 0.179104 Oct 13 04:22:36.194611 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 129/231] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:36 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1515 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:36.312794 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:36.313528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:36.319765 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:36.320410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:36.321117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:36.322286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:36.322890 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:36.323541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:36.489468 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:36.499593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-24421a7e-e1b7-4d01-a0b9-468cdb791b85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.190745 Oct 13 04:22:36.500279 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 103/232] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:36 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:37.525997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:37.526633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:37.533796 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:37.534515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:37.534945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:37.535685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:37.537276 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:37.537708 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:37.681804 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:37.684297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:37.693446 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:37.693934 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:37.694462 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:37.695655 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:37.696273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:37.696928 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:37.718696 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:37.727776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4028f50d-5b78-47db-9a18-33b661ef0ae1 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.205822 Oct 13 04:22:37.742229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'flavor' on Instance uuid 68b9a887-b69f-4e94-b26b-95490a16f5f9 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:22:37.742902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 130/233] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:37 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:37.863203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3fd5c03d-ae2a-403f-9446-789b3d322673 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/action" status: 200 len: 32 microversion: 2.1 time: 0.185881 Oct 13 04:22:37.864096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 104/234] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:22:37 2022] POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/action => generated 32 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 13 04:22:37.874311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:37.874921 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:37.882770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:37.883340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:37.883797 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:37.884491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:37.885200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:37.885621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:38.071439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:38.085176 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ed07e0e5-7a17-406a-a69c-6d45ed0d5353 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.215494 Oct 13 04:22:38.086074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 131/235] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:37 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:38.756476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:38.757291 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:38.764514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:38.765761 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:38.766417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:38.767344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:38.767798 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:38.768224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:38.937582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:38.945154 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0ff2c54a-043a-4ad9-95ca-c58485ecd821 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.195904 Oct 13 04:22:38.945725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 105/236] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:38 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:39.109693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:39.110295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:39.117355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:39.117593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:39.118088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:39.118861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:39.119227 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:39.119732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:39.196823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-ab69a210-bef5-4a94-8dc2-3d45502ab3d6 req-15910c1d-a62c-44cb-ad0b-1ae15f3b7a1f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1af8fd18-6186-4269-aa97-d5bc623c9b7f", "name": "network-vif-plugged", "status": "completed", "tag": "ad559ec1-ddc7-418a-ab7d-758dffdd11af"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:39.206648 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ab69a210-bef5-4a94-8dc2-3d45502ab3d6 req-15910c1d-a62c-44cb-ad0b-1ae15f3b7a1f service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:39.207134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ab69a210-bef5-4a94-8dc2-3d45502ab3d6 req-15910c1d-a62c-44cb-ad0b-1ae15f3b7a1f service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:39.207567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ab69a210-bef5-4a94-8dc2-3d45502ab3d6 req-15910c1d-a62c-44cb-ad0b-1ae15f3b7a1f service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:39.220534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-ab69a210-bef5-4a94-8dc2-3d45502ab3d6 req-15910c1d-a62c-44cb-ad0b-1ae15f3b7a1f service nova] Creating event network-vif-plugged:ad559ec1-ddc7-418a-ab7d-758dffdd11af for instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f on hv-cinder-86099 Oct 13 04:22:39.228758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-ab69a210-bef5-4a94-8dc2-3d45502ab3d6 req-15910c1d-a62c-44cb-ad0b-1ae15f3b7a1f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033567 Oct 13 04:22:39.229851 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 106/237] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:22:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:22:39.297622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:39.306514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ff77ab3f-7d99-4938-851c-9ff2e5bd26b2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.201720 Oct 13 04:22:39.307300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 132/238] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:39 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:39.969224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:39.969913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:39.976063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:39.976611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:39.976916 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:39.977597 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:39.978498 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:39.978996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:40.119480 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:40.128490 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7d9f661c-e643-4b77-a86a-f157e8de9236 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.163872 Oct 13 04:22:40.129170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 107/239] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:39 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:40.329910 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:40.330651 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:40.336769 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:40.337285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:40.337724 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:40.338441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:40.338927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:40.339350 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:40.486800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:40.490376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:40.494956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-96cdd380-1c2c-4add-a381-fa793c63ae79 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.169306 Oct 13 04:22:40.495526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 133/240] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:40 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:41.151428 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:41.152034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:41.159101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:41.159590 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:41.160015 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:41.160705 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:41.161157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:41.161574 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:41.299758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:41.307374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-59edb10e-8c41-4ec1-815c-de4bae5febc7 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1314 microversion: 2.1 time: 0.159954 Oct 13 04:22:41.308255 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 108/241] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:41 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:41.520580 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:41.521370 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:41.527693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:41.528106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:41.528638 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:41.532720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:41.533248 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:41.533816 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:41.739805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:41.749200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-374b7ea9-6db5-4f7d-9cca-ab96e0b96c2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.233697 Oct 13 04:22:41.754665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 134/242] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:41 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:42.332430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:42.333058 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:42.339481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:42.339959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:42.340385 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:42.341711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:42.342426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:42.342851 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:42.469189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:42.476758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6031f480-96bb-42e3-8e1d-4d565f82692d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1498 microversion: 2.1 time: 0.148170 Oct 13 04:22:42.477316 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 109/243] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:42 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1498 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:42.772755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:42.773361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:42.779469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:42.779950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:42.780379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:42.781062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:42.781513 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:42.781930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:42.926051 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:42.933654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5592d64d-dfd4-4881-a952-39f1973824e8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.164762 Oct 13 04:22:42.934234 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 135/244] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:42 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:43.957195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:43.957801 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:43.964229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:43.964713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:43.965146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:43.965826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:43.967021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:43.967389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:44.096846 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:44.104389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-992fcf76-1650-4cfd-ae50-4d81cb60002c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.150967 Oct 13 04:22:44.104963 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 110/245] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:43 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:45.126930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:45.127578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:45.133438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:45.133919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:45.134391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:45.135140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:45.135601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:45.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:45.261409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:45.270180 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-57224979-7a01-4fea-9e17-d48028c37e68 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.147066 Oct 13 04:22:45.270786 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 136/246] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:45 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:45.839402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:45.839894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:45.846879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:45.847362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:45.847790 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:45.848505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:45.848950 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:45.849365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:45.874824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:22:45.956491 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ea3d356e-e53f-426d-98cd-bc1db3dcb2d5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "DELETE /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 204 len: 0 microversion: 2.1 time: 0.120571 Oct 13 04:22:45.957084 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 111/247] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:22:45 2022] DELETE /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:22:45.965458 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:45.966063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:45.971957 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:45.972437 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:45.972865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:45.973568 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:45.973999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:45.974479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:46.108129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:46.115825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-096529e5-3b61-4e7e-b30a-dbf37ae41de8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1504 microversion: 2.1 time: 0.154171 Oct 13 04:22:46.116401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 137/248] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:45 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1504 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:46.294309 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:46.295043 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:46.300985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:46.301472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:46.301906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:46.302623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:46.303078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:46.303504 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:46.321976 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:46.437937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:46.446087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-45fae33a-0dc1-4dfa-af38-f943b85b08c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.155901 Oct 13 04:22:46.446691 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 112/249] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:46 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:47.139400 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:47.140104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:47.146492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:47.147054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:47.147497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:47.148190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:47.148643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:47.149067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:47.300160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:47.309558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5fb58298-1915-461a-b845-ef140ae3e1bf tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1504 microversion: 2.1 time: 0.174445 Oct 13 04:22:47.310130 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 138/250] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:47 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1504 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:47.471388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:47.472016 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:47.478365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:47.478845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:47.479268 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:47.479934 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:47.480377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:47.480789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:47.611065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:47.618593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-509d2b40-9dc2-4923-ac3a-c0a508b109b6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.151857 Oct 13 04:22:47.619153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 113/251] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:47 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:48.336878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:48.337485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:48.339346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:22:48.345608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:48.346092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:48.346557 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:48.347244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:48.347715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:48.348130 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:48.641234 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:48.641839 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:48.648216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:48.648704 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:48.649134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:48.649827 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:48.650299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:48.650715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:48.684218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:48.692853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-51d0310b-88bb-4923-a734-a959303ac7c5 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1504 microversion: 2.1 time: 0.359762 Oct 13 04:22:48.693427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 139/252] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:48 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1504 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:48.779265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:48.786890 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-073fc85c-60a2-463b-9f58-4a305750a0c6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.149851 Oct 13 04:22:48.787458 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 114/253] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:48 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:49.716211 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:49.716754 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:49.722636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:49.723120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:49.723558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:49.724240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:49.724694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:49.725120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:49.815828 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:49.816474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:49.823717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:49.823929 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:49.824319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:49.825541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:49.826797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:49.827237 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:49.855875 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:49.864414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1b0de6e7-eeba-4714-8195-408853725310 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1504 microversion: 2.1 time: 0.151865 Oct 13 04:22:49.865064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 140/254] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:49 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1504 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:49.956311 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:49.963848 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5ffcb92e-a06f-48b2-8f53-1f42dc3edb40 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.158978 Oct 13 04:22:49.964408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 115/255] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:49 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:50.888179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:50.888984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:50.895026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:50.895509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:50.895939 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:50.896630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:50.897079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:50.897498 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:50.962346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:50.971113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dfb5b75c-7d45-4b55-9c03-6497fb81d134 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 200 len: 1462 microversion: 2.1 time: 0.086903 Oct 13 04:22:50.971688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 141/256] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:50 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 1462 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:50.988823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:50.989562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:51.003759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:51.005001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:51.005534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:51.006359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:51.006879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:51.007374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:51.053936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-d4afa50d-bfb5-4864-ba63-63edd1f4b1cd req-01b43fc4-48c7-496b-9b93-2741f688a548 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1af8fd18-6186-4269-aa97-d5bc623c9b7f", "name": "network-vif-deleted", "tag": "ad559ec1-ddc7-418a-ab7d-758dffdd11af"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:51.066083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d4afa50d-bfb5-4864-ba63-63edd1f4b1cd req-01b43fc4-48c7-496b-9b93-2741f688a548 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:51.066968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d4afa50d-bfb5-4864-ba63-63edd1f4b1cd req-01b43fc4-48c7-496b-9b93-2741f688a548 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:51.067396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d4afa50d-bfb5-4864-ba63-63edd1f4b1cd req-01b43fc4-48c7-496b-9b93-2741f688a548 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:51.082279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-d4afa50d-bfb5-4864-ba63-63edd1f4b1cd req-01b43fc4-48c7-496b-9b93-2741f688a548 service nova] Creating event network-vif-deleted:ad559ec1-ddc7-418a-ab7d-758dffdd11af for instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f on hv-cinder-86099 Oct 13 04:22:51.089751 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-d4afa50d-bfb5-4864-ba63-63edd1f4b1cd req-01b43fc4-48c7-496b-9b93-2741f688a548 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040200 Oct 13 04:22:51.090505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 142/257] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:22:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:22:51.286344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:51.293972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-24ffad23-2269-4086-a382-ccb68b4daed3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.310404 Oct 13 04:22:51.295195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 116/258] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:50 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:51.998832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:51.999813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 1af8fd18-6186-4269-aa97-d5bc623c9b7f] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:52.006100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:52.006605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:52.007032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:52.007712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:52.008162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:52.008576 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:52.022961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] HTTP exception thrown: Instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f could not be found. Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Request method failure captured: Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f HTTP/1.1 Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5J52o-7CFNX8ioXczmWExnn9nKAHe0yeP0vld-0egnUWjyWi3oLgHz_pfo-eFdd9giI3Cp5alTXK_ntnozRUIg7c4nWisdzVan0m-qgG8EqYGVFIRq0wcpIU39JsGB801KhB5IRxtRei9prF1hW6WE7FG1JGsJl-cM3vz-m1QahGMg Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: e7d1e420eec84d1f8b78333b7d814f3c Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesActionsTest-1853970953 Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:22:52.024635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/e7d1e420eec84d1f8b78333b7d814f3c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/e7d1e420eec84d1f8b78333b7d814f3c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/e7d1e420eec84d1f8b78333b7d814f3c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_e7d1e420eec84d1f8b78333b7d814f3c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesActionsTest-1853970953 Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: e7d1e420eec84d1f8b78333b7d814f3c Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesActionsTest-1853970953 Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesActionsTest-1853970953-project Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 9ee6eac2e6da4d83bd14324669dc7269 Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesActionsTest-1853970953-project Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f could not be found. Oct 13 04:22:52.026238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '1af8fd18-6186-4269-aa97-d5bc623c9b7f'} Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f could not be found. Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:22:52.027174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f could not be found. Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f could not be found. Oct 13 04:22:52.028355 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:22:52.029988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Returning 404 to user: Instance 1af8fd18-6186-4269-aa97-d5bc623c9b7f could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:22:52.029988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0042331c-02b2-435f-b57f-a225da2b2c79 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f" status: 404 len: 111 microversion: 2.1 time: 0.032897 Oct 13 04:22:52.029988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 143/259] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:51 2022] GET /compute/v2.1/servers/1af8fd18-6186-4269-aa97-d5bc623c9b7f => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:22:52.142128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-591676975", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "networks": [{"uuid": "7358a795-2926-4538-a53c-de076264b109"}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:52.196151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] validate_networks() for [('7358a795-2926-4538-a53c-de076264b109', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:22:52.316756 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:52.317360 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:52.323409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:52.323881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:52.324303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:52.324980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:52.325421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:52.325865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:52.461794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:52.469952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b2ca627f-6257-4aaa-b83d-a7766c175b15 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.156915 Oct 13 04:22:52.470635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 144/260] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:52 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:52.549422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:22:52.549845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:22:52.550356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:22:52.550766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:22:52.551178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:22:52.559836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:52.560994 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:52.561422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:52.562144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:52.562619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:52.563036 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:52.579669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Getting quotas for project e7d1e420eec84d1f8b78333b7d814f3c. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:22:52.583634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Getting quotas for user 9ee6eac2e6da4d83bd14324669dc7269 and project e7d1e420eec84d1f8b78333b7d814f3c. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:22:52.595088 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:22:52.597330 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:22:52.643471 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-996235ab-150e-450f-82e6-072a04c12192 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.505544 Oct 13 04:22:52.644108 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 117/261] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:22:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 507 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 13 04:22:52.656956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7f09b3e6-c108-4ab6-b2a3-dd3a05cad87d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:52.657657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7f09b3e6-c108-4ab6-b2a3-dd3a05cad87d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:52.720268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7f09b3e6-c108-4ab6-b2a3-dd3a05cad87d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:52.732695 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7f09b3e6-c108-4ab6-b2a3-dd3a05cad87d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1213 microversion: 2.1 time: 0.080774 Oct 13 04:22:52.733275 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 145/262] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:52 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:53.495509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:53.496111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:53.503587 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:53.504068 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:53.504500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:53.505178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:53.505628 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:53.506047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:53.634984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:53.643412 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ff3561a5-a937-43e0-bc5f-f9ed6408771e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.152624 Oct 13 04:22:53.643972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 118/263] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:53 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:53.756735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:53.757337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:53.763915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:53.764669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:53.765099 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:53.765777 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:53.766247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:53.766668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:53.825362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:53.833592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6f3a89d0-9096-4fc8-86f4-1afa9621e42d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1217 microversion: 2.1 time: 0.080638 Oct 13 04:22:53.834153 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 146/264] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:53 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1217 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:54.666917 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:54.667554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:54.674247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:54.674681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:54.675113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:54.675822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:54.676274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:54.676695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:54.802515 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:54.810828 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-97a1d03f-52e7-444d-bed4-b70d33874a4f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.147215 Oct 13 04:22:54.811393 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 119/265] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:54 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:54.855692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:54.857357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:54.863229 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:54.863714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:54.864145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:54.864848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:54.865300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:54.865721 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:55.067705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:55.075176 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0cc3f964-6e5b-4677-a35b-d3c5e8e7be97 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1217 microversion: 2.1 time: 0.223268 Oct 13 04:22:55.082366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 147/266] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:54 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1217 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:55.832745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:55.833495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:55.839979 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:55.840462 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:55.840896 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:55.842346 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:55.842807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:55.843266 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:55.974178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:55.982545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-df9bdde4-b783-4c35-b4ea-c67bfcb51ca1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.153657 Oct 13 04:22:55.984017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 120/267] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:55 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:56.103553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:56.104152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:56.110565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:56.111046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:56.111473 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:56.112153 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:56.112598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:56.113011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:56.178520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:56.187250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1807a7c5-7ac4-43bb-a4f7-7cbfdd38e9c0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1265 microversion: 2.1 time: 0.087427 Oct 13 04:22:56.187886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 148/268] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:56 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1265 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:57.007811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:57.008681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:57.018166 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:57.019062 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:57.019598 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:57.020522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:57.021196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:57.021821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:57.212516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:57.213130 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:57.220830 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:57.221958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:57.222452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:57.223913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:57.224536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:57.224963 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:57.369158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:57.378264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6eb999c8-99c6-4448-863f-2ffd67547666 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.169511 Oct 13 04:22:57.378821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 149/269] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:57 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:57.443412 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:57.453871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-adbdfe28-ceb9-4095-a86a-732b3e571053 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.450937 Oct 13 04:22:57.454561 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 121/270] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:57 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:58.150641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-cb565376-c0f9-4125-ad5f-5298bfd336ce req-8804a377-f98a-4585-a177-0bbe4c8b5064 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "676959ea-2e7e-4410-aec0-645c6d2b078a", "tag": "2e7828d5-f322-47d7-8c6b-69e006c676b6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:22:58.178725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-cb565376-c0f9-4125-ad5f-5298bfd336ce req-8804a377-f98a-4585-a177-0bbe4c8b5064 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:58.179206 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-cb565376-c0f9-4125-ad5f-5298bfd336ce req-8804a377-f98a-4585-a177-0bbe4c8b5064 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:58.179638 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-cb565376-c0f9-4125-ad5f-5298bfd336ce req-8804a377-f98a-4585-a177-0bbe4c8b5064 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:58.190296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-cb565376-c0f9-4125-ad5f-5298bfd336ce req-8804a377-f98a-4585-a177-0bbe4c8b5064 service nova] Creating event network-changed:2e7828d5-f322-47d7-8c6b-69e006c676b6 for instance 676959ea-2e7e-4410-aec0-645c6d2b078a on hv-cinder-86099 Oct 13 04:22:58.195659 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-cb565376-c0f9-4125-ad5f-5298bfd336ce req-8804a377-f98a-4585-a177-0bbe4c8b5064 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048683 Oct 13 04:22:58.196243 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 150/271] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:22:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:22:58.402899 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:58.404479 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:58.412305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:58.412305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:58.413232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:58.413406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:58.413963 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:58.414538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:58.480384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:58.482830 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:58.492375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:58.493122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:58.493720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:58.494886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:58.495493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:58.496075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:58.578252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:58.586506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fed6735d-cdce-4b7a-b5ab-b274d7d53a06 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.189318 Oct 13 04:22:58.587188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 122/272] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:58 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:58.795969 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:58.808157 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2692fe74-462e-4c5a-a468-c6fa2a4e440d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.332072 Oct 13 04:22:58.809230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 151/273] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:58 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:22:59.537284 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:59.537871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:59.545678 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:59.545837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:59.546379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:59.547157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:59.547676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:59.548164 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:59.573414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8d698265-7344-4443-8c8b-a42b639dc552 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/ips" status: 200 len: 150 microversion: 2.1 time: 0.040051 Oct 13 04:22:59.573996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 123/274] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:22:59 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/ips => generated 150 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:22:59.609627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:59.610851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:59.616663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:59.617146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:59.617741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:59.618471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:59.618928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:59.619353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:59.836188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:22:59.842195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:22:59.846770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:22:59.860831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-622b7cbb-1410-48bf-8d27-b91e077ca385 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.255834 Oct 13 04:22:59.861705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 152/275] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:59 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:22:59.863214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:59.869169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:59.869971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:22:59.872633 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:22:59.884903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:22:59.885602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:00.129385 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:00.137519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bcf04690-bfd8-425e-b0e6-50dd95b9144d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.308323 Oct 13 04:23:00.151273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 124/276] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:22:59 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:00.882727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:00.883324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:00.890676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:00.891159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:00.895544 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:00.899718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:00.900409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:00.901016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:00.932176 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:01.151396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:01.166254 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:01.166873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:01.173155 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b5025274-f9d5-4ecb-8583-37a80e032f05 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.293248 Oct 13 04:23:01.178019 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 153/277] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:00 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:01.180919 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:01.183440 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:01.184195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:01.185781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:01.186849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:01.187517 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:01.188154 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:01.367294 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:01.380651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-781df04a-e77c-4e0d-9b2e-0b1f85004cdc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.219563 Oct 13 04:23:01.381838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 125/278] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:01 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:02.135392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ccd0cb13-8965-4bc8-a8c4-3c35c2995160"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:02.138895 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:02.152104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:02.154892 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:02.155434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:02.156180 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:02.156655 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:02.159376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:02.197899 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:02.198541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:02.225836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:02.226363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:02.226803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:02.227514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:02.235990 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:02.237087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:02.493792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:02.503310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-413e83a5-718f-4aa0-95f5-18a51d2504d9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.309423 Oct 13 04:23:02.503993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 126/279] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:02 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:02.511813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:02.513807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:02.520760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:02.521613 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:02.522157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:02.523067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:02.523763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:02.524204 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:02.570840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:02.590399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:02.674143 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:02.681924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-27a6b37c-42ed-43d1-b4d2-d1632dce800b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.176888 Oct 13 04:23:02.682522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 127/280] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:02 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:02.729348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:03.529857 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:03.531250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:03.541096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:03.550082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:03.550082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:03.550082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:03.550082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:03.550766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:03.817123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:03.848611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ecc3712e-bd92-44a5-af5f-c5c4c31b265d tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.323406 Oct 13 04:23:03.849236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 128/281] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:03 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:03.878165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:03.879732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:03.886453 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:03.895677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-99353a0c-7d2a-4ab7-825f-87442dbca8e6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.771299 Oct 13 04:23:03.896764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 154/282] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:23:02 2022] POST /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments => generated 194 bytes in 1774 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:03.897315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:03.900945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:03.901958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:03.909325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:03.913210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:04.144012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:04.155813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-258a8a24-cd19-4209-88e0-a962682b28a1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.289456 Oct 13 04:23:04.156492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 129/283] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:03 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:04.879176 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:04.884546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:04.894842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:04.897899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:04.899472 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:04.900382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:04.902755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:04.903703 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:05.180286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:05.181513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:05.195938 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:05.196806 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:05.197578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:05.198578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:05.199198 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:05.199771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:05.205460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:05.229597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:05.240543 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-38390c2d-5c0d-44cd-b0b5-d41952638d85 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.365807 Oct 13 04:23:05.241337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 155/284] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:04 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:05.384094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:05.394932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b1a09bd8-88fb-4f67-a06a-02c07f39cb09 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.218608 Oct 13 04:23:05.395532 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 130/285] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:05 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:05.886440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:05.888696 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:05.908349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:05.909024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:05.909465 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:05.913941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:05.914407 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:05.915772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:05.956342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fe9b9346-aa78-4548-b52a-f88bbdc45050 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/ips" status: 200 len: 152 microversion: 2.20 time: 0.074582 Oct 13 04:23:05.958254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 156/286] 10.222.0.27 () {60 vars in 1289 bytes} [Thu Oct 13 04:23:05 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/ips => generated 152 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:23:06.266157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:06.266796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:06.275854 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:06.276363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:06.276800 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:06.277512 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:06.277970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:06.286659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:06.418719 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:06.420068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:06.436709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:06.437526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:06.438102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:06.439033 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:06.439629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:06.440347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:06.707720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:06.718442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-be826e3b-c24e-4f53-81d4-023b5a7b6cf6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.303499 Oct 13 04:23:06.730307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 157/287] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:06 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:06.748709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:06.758055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-49e8ecb4-b0bb-4ad4-b43e-7c11ab9d3492 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.496397 Oct 13 04:23:06.758713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 131/288] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:06 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:07.746157 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:07.747514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:07.756594 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:07.757397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:07.757939 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:07.759259 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:07.759829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:07.761356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:07.782640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:07.784043 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:07.804799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:07.806883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:07.810671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:07.811854 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:07.812322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:07.812756 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:08.054424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:08.077571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5b6c6b60-4189-4c30-93b6-1814bd65ec37 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.333667 Oct 13 04:23:08.098471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 158/289] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:07 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:08.227502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:08.237206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0c7b5718-9cc1-46e5-af63-c663daf136e9 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.460086 Oct 13 04:23:08.254764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 132/290] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:07 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:09.021770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-57f8b8f0-a6cc-4e5e-b789-8690d353e4b4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Action: 'create', calling method: .version_select at 0x7f0794b14940>, body: {"keypair": {"name": "tempest-TestShelveInstance-243333657", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFyq7PR380wcg6EmVGU7JJKX+dhQMUlUVdqKHVBh+rP3puoz5ihSqOKJ7YssETkl9KdVanuO3bCC8p1c7WBY2FgvTdRPawGmH5OF8wlAOaCHAmDF16klrfeGtaR8A6+mNA=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:09.039612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-57f8b8f0-a6cc-4e5e-b789-8690d353e4b4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Getting quotas for project fabffb6796a34773987a4f7e414352d0. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:23:09.044715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-57f8b8f0-a6cc-4e5e-b789-8690d353e4b4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Getting quotas for user 242490d937324b31bc95ae1982ea0b39 and project fabffb6796a34773987a4f7e414352d0. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:23:09.069067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-57f8b8f0-a6cc-4e5e-b789-8690d353e4b4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.228922 Oct 13 04:23:09.069806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 159/291] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:23:08 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:09.110998 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:09.112258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:09.122913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.123680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.124276 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.125153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.125763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.126398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.168460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:09.170123 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:09.190245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.190961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.191668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.194043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.194946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.196835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.271668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lazy-loading 'flavor' on Instance uuid cc378324-badd-4a63-ba04-4a02fe6551f8 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:23:09.624376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-20d6915e-118f-440f-b5ae-5fec77e3a3d6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/action" status: 202 len: 0 microversion: 2.20 time: 0.461634 Oct 13 04:23:09.626525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 160/292] 10.222.0.27 () {62 vars in 1319 bytes} [Thu Oct 13 04:23:09 2022] POST /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/action => generated 0 bytes in 467 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:23:09.640651 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:09.643327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:09.660839 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:09.663372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.665428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.666115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.667090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.667692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.668282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.678629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-958c4f1b-1e5b-43a8-a03b-c11cccb3600e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1501 microversion: 2.1 time: 0.575308 Oct 13 04:23:09.679223 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 133/293] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:09 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1501 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:09.688434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:09.689274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:09.696373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.697082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.697811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.698789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.699428 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.700054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.906165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:09.918720 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:09.920939 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:09.923284 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-128c37e6-bb19-4476-b492-b0868f2f59b0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.242589 Oct 13 04:23:09.938231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 134/294] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:09 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:09.938231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0ede059a-2d95-46c3-b249-480498cdbe00"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:09.938231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:09.945525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.946937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.947322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.948478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:09.948952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:09.949386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:09.953216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:09.975390 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-12432920-f4df-40f9-9ec4-e9331c5651a6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.338412 Oct 13 04:23:09.976042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 161/295] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:09 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:10.002481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] HTTP exception thrown: Cannot 'attach_volume' instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 while it is in vm_state rescued Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Request method failure captured: Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/os-volume_attachments HTTP/1.1 Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 74 Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I1Q5IsW5tb9caZYjTnxg76OHJrIRxhA2IsmP-4bjo-Sp8a3KWoY0-TA9ATSJXlQ7ulS9_F3063hs4H1CC9BtmBLHeEh6xOijc0lcEWl3ixN4pgW7TWB892vj9IX631a7RX42s1OkuJTtVHG7SznQ4EGjFdvEIILnlDfDtCnOPDW0s Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:23:10.004457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/0913f8eb06164012b366d62c553d261c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_0913f8eb06164012b366d62c553d261c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: da9c75191a3b4ef49438aae9e35fcf13 Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {"volumeAttachment": {"volumeId": "0ede059a-2d95-46c3-b249-480498cdbe00"}} Oct 13 04:23:10.005685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Cannot 'attach_volume' instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 while it is in vm_state rescued Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'server_id': '68b9a887-b69f-4e94-b26b-95490a16f5f9', 'body': {'volumeAttachment': {'volumeId': '0ede059a-2d95-46c3-b249-480498cdbe00'}}} Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:23:10.006437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 while it is in vm_state rescued Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:23:10.007500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 while it is in vm_state rescued Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Returning 409 to user: Cannot 'attach_volume' instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 while it is in vm_state rescued {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7c8a30fa-d723-48fd-a029-f0da130cd8c0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.080045 Oct 13 04:23:10.008510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 135/296] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:23:09 2022] POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/os-volume_attachments => generated 152 bytes in 82 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 13 04:23:10.019315 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:10.020158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:10.036376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:10.036982 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:10.039226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:10.042994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:10.044484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:10.046656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:10.151336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-00f94f53-cbfa-41ad-8bc1-e4c82a664d17 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/action" status: 202 len: 0 microversion: 2.1 time: 0.136448 Oct 13 04:23:10.152766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 162/297] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:23:10 2022] POST /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9/action => generated 0 bytes in 138 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:23:10.162478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:10.163258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:10.174499 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:10.175002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:10.175449 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:10.178948 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:10.180317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:10.181876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:10.379496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:10.388455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5047b4c3-1840-4b67-bcaf-319c7558d405 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1509 microversion: 2.1 time: 0.231169 Oct 13 04:23:10.389352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 136/298] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:10 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1509 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:10.955561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:10.956333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:10.962832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:10.963236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:10.963665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:10.964459 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:10.964844 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:10.965384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:11.003974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:11.014971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:11.023772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:11.034868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:11.035508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:11.036357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:11.036954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:11.037480 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:11.185345 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:11.203985 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:11.205567 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:11.207008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ec59c3e8-2b2c-4533-a749-07ed7bdef5ea tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.258032 Oct 13 04:23:11.207867 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 163/299] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:10 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:11.279422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:11.288074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-91445633-d4c3-4d39-a220-73fd50d8250c tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.287964 Oct 13 04:23:11.288644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 137/300] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:10 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:11.416056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:11.416945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:11.423419 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:11.423909 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:11.424515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:11.425268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:11.425733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:11.426159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:11.560989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:11.568694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-65a38e55-24e4-4faa-a2e1-290824cecf16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1509 microversion: 2.1 time: 0.156972 Oct 13 04:23:11.569247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 164/301] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:11 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1509 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:12.236657 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:12.237359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:12.244439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:12.245037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:12.245536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:12.246337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:12.247499 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:12.247996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:12.312787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:12.313390 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:12.321292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:12.322007 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:12.322608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:12.323452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:12.324169 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:12.324777 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:12.422765 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:12.432156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:12.434293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:12.435888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-40197de1-1532-4911-b2bb-f867f74a8334 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.206708 Oct 13 04:23:12.437145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 138/302] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:12 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:12.505357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:12.520497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fa12580c-33f1-485e-96c8-7717df115991 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.212759 Oct 13 04:23:12.534235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 165/303] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:12 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:12.595056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:12.595838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:12.603773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:12.604399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:12.604960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:12.606531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:12.607811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:12.608341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:12.806971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:12.816775 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cd9803c1-8e85-4f2d-b978-e382305a7909 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1509 microversion: 2.1 time: 0.226408 Oct 13 04:23:12.817541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 139/304] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:12 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1509 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:13.464736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:13.466655 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:13.473424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:13.474103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:13.474665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:13.475357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:13.475972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:13.476441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:13.545551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:13.546300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:13.553699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:13.554510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:13.555013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:13.555783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:13.556447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:13.557065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:13.682469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:13.695697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:13.698395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:13.700453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c432b53f-fc1d-4c0f-a983-e88142a68188 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.239349 Oct 13 04:23:13.701328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 166/305] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:13 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:13.759131 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:13.774838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e7f1bef9-4a66-46a5-8100-8aff8336baac tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.233889 Oct 13 04:23:13.775431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 140/306] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:13 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:13.841375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:13.842077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:13.849579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:13.850298 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:13.850885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:13.851755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:13.852515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:13.853258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:13.862669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:14.038745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:14.047529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-11f19436-135f-45a2-ab7e-9bac0bc0ef2e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1509 microversion: 2.1 time: 0.210829 Oct 13 04:23:14.054273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 167/307] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:13 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1509 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:14.727953 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:14.728560 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:14.735903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:14.736571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:14.737008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:14.738086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:14.738616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:14.739046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:14.806071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:14.806733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:14.813962 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:14.814487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:14.814922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:14.815764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:14.816222 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:14.816645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:14.897519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:14.907386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:14.909328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:14.911216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-32355354-c910-4977-9d81-79cb973a7d98 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.187114 Oct 13 04:23:14.912030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 141/308] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:14 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:14.993302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:15.002423 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1205f275-3ce7-4f54-afea-02e1aadab75f tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.204148 Oct 13 04:23:15.003300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 168/309] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:14 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:15.073685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:15.074336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:15.083977 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.084282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.084820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:15.085573 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.086068 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.086578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:15.150428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-5c2f07cb-75a1-416c-a24a-70f2c1ef1d7c req-d2acfe58-d835-47ca-9517-91d9d4c059eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "676959ea-2e7e-4410-aec0-645c6d2b078a", "name": "network-vif-plugged", "status": "completed", "tag": "2e7828d5-f322-47d7-8c6b-69e006c676b6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:15.163645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5c2f07cb-75a1-416c-a24a-70f2c1ef1d7c req-d2acfe58-d835-47ca-9517-91d9d4c059eb service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.164476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5c2f07cb-75a1-416c-a24a-70f2c1ef1d7c req-d2acfe58-d835-47ca-9517-91d9d4c059eb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.165011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-5c2f07cb-75a1-416c-a24a-70f2c1ef1d7c req-d2acfe58-d835-47ca-9517-91d9d4c059eb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:15.177492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-5c2f07cb-75a1-416c-a24a-70f2c1ef1d7c req-d2acfe58-d835-47ca-9517-91d9d4c059eb service nova] Creating event network-vif-plugged:2e7828d5-f322-47d7-8c6b-69e006c676b6 for instance 676959ea-2e7e-4410-aec0-645c6d2b078a on hv-cinder-86099 Oct 13 04:23:15.182858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-5c2f07cb-75a1-416c-a24a-70f2c1ef1d7c req-d2acfe58-d835-47ca-9517-91d9d4c059eb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037127 Oct 13 04:23:15.183449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 169/310] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:23:15.242802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:15.253055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aa9d9d71-f1e9-48e4-a516-c2d1ba397897 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1509 microversion: 2.1 time: 0.183563 Oct 13 04:23:15.254295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 142/311] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:15 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1509 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:15.573023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:15.573601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:15.580606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.581083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.581679 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:15.582395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.582840 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.583254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:15.606480 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a9962189-fe6f-4ee8-9e30-1c1f02606734 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/ips" status: 200 len: 145 microversion: 2.1 time: 0.037351 Oct 13 04:23:15.607406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 170/312] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:23:15 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/ips => generated 145 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:15.940873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:15.941461 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:15.954006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.954681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.955121 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:15.956021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:15.956623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:15.957356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:16.034848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:16.038329 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:16.053878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:16.055077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:16.064625 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:16.065574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:16.066598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:16.067276 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:16.263356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:16.275400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:16.276830 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:16.279691 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-38c480f7-ec61-406f-8439-5e416fede6ac tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.343116 Oct 13 04:23:16.282187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 143/313] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:15 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:16.287230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:16.288126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:16.295185 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:16.295850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:16.296668 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:16.297360 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:16.297966 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:16.298417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:16.299301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:16.307036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-125ec00d-f487-4f36-9b1d-c680561597bb tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.284988 Oct 13 04:23:16.307760 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 171/314] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:16 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:16.426869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:16.435409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-68bb16f6-1396-4823-9245-e6c5702eb168 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1515 microversion: 2.1 time: 0.151975 Oct 13 04:23:16.436085 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 144/315] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:16 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:17.144745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d4e76def-0741-4131-8a36-5438b0f7c6ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: .version_select at 0x7f0794ab65e0>, body: {"keypair": {"name": "tempest-keypair-715085596", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBfyczQj1lU1P4hFwzNaVZv6meTHJv8+2TqepOFTEwPswXbdFRoUD0hvMEx8VWb2DKBBRiwx03am3Dr/9bAOX7YQ6wC7owBsc7YntgXdw6R2DKVhoYJ43ExkGDY6c7uimA=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:17.161575 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-d4e76def-0741-4131-8a36-5438b0f7c6ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:23:17.165538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-d4e76def-0741-4131-8a36-5438b0f7c6ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:23:17.190008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d4e76def-0741-4131-8a36-5438b0f7c6ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.048929 Oct 13 04:23:17.190662 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 172/316] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:23:17 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:17.307180 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:17.310180 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:17.320065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:17.320851 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:17.321452 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:17.322819 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:17.323439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:17.325212 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:17.331806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:17.332556 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:17.342042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:17.342733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:17.343310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:17.344310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:17.344991 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:17.346082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:17.556418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:17.566169 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e88e569e-eac0-49e0-8daa-d09fdd875ae8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.238707 Oct 13 04:23:17.585373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 173/317] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:17 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:17.614461 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:17.645122 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:17.647311 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:17.648982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-485ec53e-544b-4d9d-8e0c-93dafca20e60 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.345918 Oct 13 04:23:17.654865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 145/318] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:17 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:17.821756 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fc8f17b8-e2e6-463c-a663-782f495a68f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'create', calling method: .version_select at 0x7f0794ad09d0>, body: {"keypair": {"name": "tempest-keypair-1691207034", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC9owCEoEjpRptIk0dFCOC6AQNplAguylLW3tUvEk1/k4dLOYtW3KPmbD9dMyjmMwfgEMe4bZFVmzv+3Uz+LmHR2GyHawpA72FEkX22r4LsE5i3y4F6DArymkFmYWgjgHQ=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:17.838167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-fc8f17b8-e2e6-463c-a663-782f495a68f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for project 0913f8eb06164012b366d62c553d261c. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:23:17.842739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-fc8f17b8-e2e6-463c-a663-782f495a68f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for user da9c75191a3b4ef49438aae9e35fcf13 and project 0913f8eb06164012b366d62c553d261c. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:23:17.851847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:17.866871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fc8f17b8-e2e6-463c-a663-782f495a68f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.049182 Oct 13 04:23:17.867660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 174/319] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:23:17 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:18.596800 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:18.597526 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:18.610793 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:18.611453 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:18.611992 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:18.612946 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:18.613484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:18.614630 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:18.686781 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:18.687410 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:18.707493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:18.710454 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:18.710573 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:18.711239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:18.711706 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:18.712627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:18.872796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:18.882774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-76cb4cbc-bea1-46b7-b42a-2e6f5352ac4b tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.292820 Oct 13 04:23:18.883892 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 146/320] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:18 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:18.892275 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:18.897611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "066597f1-58d0-42ea-bc0f-de4e204ffa0c"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:18.900513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:18.904524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:18.908544 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:18.909174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:18.909616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:18.910582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:18.910702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:18.910976 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:18.911557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:18.912134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-993d6a19-3e15-426e-993e-efc1b0137c6b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.239546 Oct 13 04:23:18.913020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 175/321] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:18 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:18.929220 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:18.952586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-299540359", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "70d105db-93f5-4592-bb80-f58af13e321b"}], "security_groups": [{"name": "tempest-securitygroup--1920818702"}], "key_name": "tempest-keypair-715085596", "user_data": "}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:19.082558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] validate_networks() for [('70d105db-93f5-4592-bb80-f58af13e321b', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:23:19.486695 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:23:19.487129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:23:19.487607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:23:19.488020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:23:19.488438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:23:19.497937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:19.498456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:19.498895 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:19.499631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:19.500083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:19.500699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:19.518810 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:23:19.523898 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:23:19.536657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:23:19.539150 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:23:19.586532 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0a997459-2101-4662-b9f4-e7d6fd31f369 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.639030 Oct 13 04:23:19.587741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 176/322] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:23:18 2022] POST /compute/v2.1/servers => generated 402 bytes in 641 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:23:19.599759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c4f73627-443e-425d-82a4-e69e5e35562f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:19.600778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c4f73627-443e-425d-82a4-e69e5e35562f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:19.682150 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c4f73627-443e-425d-82a4-e69e5e35562f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:19.690739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c4f73627-443e-425d-82a4-e69e5e35562f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1240 microversion: 2.1 time: 0.095134 Oct 13 04:23:19.691871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 177/323] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:19 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1240 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:19.790864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1981895619", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "618621b6-9258-44fe-b6fb-530eb2b4c8f1"}], "security_groups": [{"name": "tempest-securitygroup--1148513686"}], "key_name": "tempest-keypair-1691207034", "user_data": "}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:19.911419 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] validate_networks() for [('618621b6-9258-44fe-b6fb-530eb2b4c8f1', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:23:20.304441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:23:20.304860 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:23:20.305331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:23:20.305736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:23:20.306145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:23:20.314953 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.315435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.315869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.316586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.317032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.317448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.335188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for project 0913f8eb06164012b366d62c553d261c. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:23:20.338866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Getting quotas for user da9c75191a3b4ef49438aae9e35fcf13 and project 0913f8eb06164012b366d62c553d261c. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:23:20.350509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:23:20.352660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:23:20.400292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c0616f84-f7d5-466f-a8b3-e31162b66fa1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.613907 Oct 13 04:23:20.401044 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 178/324] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:23:19 2022] POST /compute/v2.1/servers => generated 409 bytes in 615 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 13 04:23:20.406351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:20.408135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:20.413342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.413932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.414613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.415237 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.415761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.416267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.553082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:20.561795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-49697f65-cba3-423f-bd5f-2b24c79a3bf0 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1313 microversion: 2.1 time: 0.159678 Oct 13 04:23:20.562389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 179/325] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:20 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:20.567607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:20.568487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:20.576941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.577469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.577924 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.578653 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.579116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.579543 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.721936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:20.729980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:20.732309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:20.733928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5db5226e-b00a-4cec-9206-8070f5c15ea5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.170551 Oct 13 04:23:20.753310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 180/326] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:20 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:20.759383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5cf379ec-17ab-45f2-94b7-20c815149799 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:20.759727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5cf379ec-17ab-45f2-94b7-20c815149799 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:20.828150 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5cf379ec-17ab-45f2-94b7-20c815149799 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:20.835019 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5cf379ec-17ab-45f2-94b7-20c815149799 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1246 microversion: 2.1 time: 0.079906 Oct 13 04:23:20.835605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 181/327] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:20 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1246 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:20.839958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:20.841103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:20.847573 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.848286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.848718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.849418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:20.849869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:20.850322 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:20.910586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:20.918600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-aeaf27f6-f985-440d-a72b-88b3ff0126e4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1244 microversion: 2.1 time: 0.081652 Oct 13 04:23:20.919193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 182/328] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:20 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1244 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:21.584005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:21.584607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:21.590553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:21.591031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:21.592110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:21.593102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:21.593557 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:21.593980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:21.741477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:21.750195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-23bd5437-fa73-4a4c-ade1-61eeff7e67b8 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1497 microversion: 2.1 time: 0.169971 Oct 13 04:23:21.758520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 183/329] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:21 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:21.765312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:21.766616 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:21.774778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:21.775556 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:21.776200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:21.777122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:21.778038 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:21.778696 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:21.942545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:21.952859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:21.954772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:21.956395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1896a907-0994-4a97-9132-4550a52b1ae5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.196157 Oct 13 04:23:21.957124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 184/330] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:21 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:21.968018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:21.968865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:21.975046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:21.975599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:21.976099 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:21.976814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:21.977269 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:21.977719 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.055017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:22.068702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-482c0cdf-6ec4-457b-8bb3-06cf9bae9c87 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1298 microversion: 2.1 time: 0.110385 Oct 13 04:23:22.069282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 185/331] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:21 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1298 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:22.082620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:22.083456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:22.090923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.092234 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.092665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.093547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.094163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.094873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.167757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:22.177173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5f754d1d-0e76-499a-9fb5-1f52789d6838 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1236 microversion: 2.1 time: 0.101712 Oct 13 04:23:22.178928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 186/332] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:22 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1236 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:22.222258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aa68e590-a310-4352-8271-3d7b0bf71b38 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.329741 Oct 13 04:23:22.226745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 147/333] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:23:18 2022] POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-volume_attachments => generated 194 bytes in 3335 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:22.677259 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:22.677756 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:22.683546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.684031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.684467 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.686092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.686579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.687056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.711609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:23:22.792977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f604979e-2c76-40a4-89af-7f00a8e37190 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "DELETE /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 204 len: 0 microversion: 2.1 time: 0.119079 Oct 13 04:23:22.793869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 187/334] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:23:22 2022] DELETE /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:23:22.804005 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:22.804697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:22.811937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.812384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.812919 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.813697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.814152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.814610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.956993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:22.967498 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3583f668-61cc-4831-8cb9-7eee6b417ea4 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1503 microversion: 2.1 time: 0.168191 Oct 13 04:23:22.968072 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 148/335] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:22 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1503 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:22.988210 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:22.988905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:22.995306 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.995786 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.996221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:22.996899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:22.997348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:22.998316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:23.017314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:23.093556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:23.094292 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:23.100407 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:23.100902 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:23.101338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:23.102055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:23.102683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:23.103118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:23.134280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:23.144320 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:23.145922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:23.147243 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ccb43c6f-3985-4a2c-9cec-83784df9eb9c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.163368 Oct 13 04:23:23.147879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 188/336] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:22 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:23.163951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:23.178941 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-451c36bd-5a78-486f-bddf-6a720e6d3e02 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1306 microversion: 2.1 time: 0.089834 Oct 13 04:23:23.181676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 149/337] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:23 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1306 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:23.205317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:23.205826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:23.211871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:23.212351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:23.212775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:23.213521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:23.213997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:23.214447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:23.276895 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:23.284829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ea7bd828-6c71-4893-a1a5-e9c71b05ac97 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1236 microversion: 2.1 time: 0.085352 Oct 13 04:23:23.285397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 189/338] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:23 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1236 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:23.992056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:23.992796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:24.000176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.000677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.001724 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.002505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.002963 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.003387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.139299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:24.147339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fa828aa9-f519-4b85-a617-b2550bea3605 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1503 microversion: 2.1 time: 0.159472 Oct 13 04:23:24.147906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 150/339] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:23 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:24.172657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:24.173850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:24.180955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.181678 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.182126 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.183133 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.183603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.184032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.204666 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:24.205364 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:24.212424 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.213098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.213662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.214808 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.215189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.215701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.291730 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:24.302997 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b1d23dfe-7354-42f0-a04c-15c088fa4c7a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1306 microversion: 2.1 time: 0.102262 Oct 13 04:23:24.303744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 151/340] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:24 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1306 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:24.322416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:24.327091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:24.343260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.344021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.344601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.346045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:24.346677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:24.347105 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:24.364708 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:24.374952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:24.378132 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:24.380994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9916984b-833d-4846-a272-a9ac3bef25a5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.211235 Oct 13 04:23:24.382182 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 190/341] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:24 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:24.451054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:24.459433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6b50d300-4a35-4b6a-89be-f7b587c04662 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1292 microversion: 2.1 time: 0.149640 Oct 13 04:23:24.460182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 152/342] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:24 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1292 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:25.172255 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:25.173695 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:25.185488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.188326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.193244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.194195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.194991 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.195512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.331051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:25.331658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:25.338307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.338791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.339218 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.339906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.340354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.340772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.382832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:25.394639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1d25ff3f-eaaf-448f-94c8-b24dd1210967 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1503 microversion: 2.1 time: 0.226797 Oct 13 04:23:25.395885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 191/343] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:25 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1503 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:25.419106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:25.420016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:25.427746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.428777 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.429528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.430299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.431143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.431771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.455900 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:25.468198 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b538dcf7-30a3-4bae-9ead-3e089706c7b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1304 microversion: 2.1 time: 0.141830 Oct 13 04:23:25.469411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 153/344] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:25 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1304 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:25.502625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:25.503288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:25.510015 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.512980 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.513825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.515508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.516336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.518259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.627753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:25.638095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:25.639584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:25.640999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dc3141c5-cf19-4197-bf56-7e5c902812d8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.226813 Oct 13 04:23:25.641626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 192/345] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:25 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:25.667990 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:25.683536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-564055c8-14a7-4fb7-a282-4c0b7d2758f4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1292 microversion: 2.1 time: 0.193653 Oct 13 04:23:25.684339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 154/346] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:25 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1292 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:25.916713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-62c8d265-c5f6-4cf2-9d97-e8bbddd7f93d req-77b497b7-3e4c-4f27-9ebe-7988c2d1228a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "676959ea-2e7e-4410-aec0-645c6d2b078a", "name": "network-vif-deleted", "tag": "2e7828d5-f322-47d7-8c6b-69e006c676b6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:25.927268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-62c8d265-c5f6-4cf2-9d97-e8bbddd7f93d req-77b497b7-3e4c-4f27-9ebe-7988c2d1228a service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:25.927725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-62c8d265-c5f6-4cf2-9d97-e8bbddd7f93d req-77b497b7-3e4c-4f27-9ebe-7988c2d1228a service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:25.928156 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-62c8d265-c5f6-4cf2-9d97-e8bbddd7f93d req-77b497b7-3e4c-4f27-9ebe-7988c2d1228a service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:25.941062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-62c8d265-c5f6-4cf2-9d97-e8bbddd7f93d req-77b497b7-3e4c-4f27-9ebe-7988c2d1228a service nova] Creating event network-vif-deleted:2e7828d5-f322-47d7-8c6b-69e006c676b6 for instance 676959ea-2e7e-4410-aec0-645c6d2b078a on hv-cinder-86099 Oct 13 04:23:25.945931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-62c8d265-c5f6-4cf2-9d97-e8bbddd7f93d req-77b497b7-3e4c-4f27-9ebe-7988c2d1228a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032851 Oct 13 04:23:25.946632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 193/347] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:23:26.421213 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:26.422127 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:26.428986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.429644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.430304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.431007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.431462 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.431882 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.477682 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.479755 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.480478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.483173 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lazy-loading 'fault' on Instance uuid 676959ea-2e7e-4410-aec0-645c6d2b078a {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:23:26.495625 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:26.496316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:26.514992 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.515740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.516527 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.517901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.518538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.519118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.640804 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9ee6eac2e6da4d83bd14324669dc7269', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7d1e420eec84d1f8b78333b7d814f3c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:26.667203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d95758f4-92df-41b4-b87a-9c91e07ce166 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 200 len: 1303 microversion: 2.1 time: 0.250256 Oct 13 04:23:26.670656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 155/348] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:26 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 1303 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:26.692606 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:26.693217 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:26.705594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.706400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.707055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.708396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.709004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.709779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.740764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:26.752175 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-156efdcc-eeea-4407-83fb-e4dbbb77c7d2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.260194 Oct 13 04:23:26.753522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 194/349] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:26 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:26.774118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:26.774457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:26.781954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.783100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.783701 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.784739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:26.785636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:26.786196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:26.862067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:26.870761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-de4e6f28-2e13-45ed-bbf0-1c97f54d87ea tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1298 microversion: 2.1 time: 0.115562 Oct 13 04:23:26.871741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 195/350] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:26 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1298 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:27.030750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:27.041856 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:27.043551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:27.044987 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-be2b479d-b3d8-4294-8427-e8cf2ced5e47 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.370110 Oct 13 04:23:27.046026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 156/351] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:26 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:27.700687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:27.702687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] [instance: 676959ea-2e7e-4410-aec0-645c6d2b078a] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:27.714619 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:27.715139 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:27.715580 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:27.716278 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:27.717760 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:27.718195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:27.736323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] HTTP exception thrown: Instance 676959ea-2e7e-4410-aec0-645c6d2b078a could not be found. Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Request method failure captured: Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a HTTP/1.1 Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5J52o-7CFNX8ioXczmWExnn9nKAHe0yeP0vld-0egnUWjyWi3oLgHz_pfo-eFdd9giI3Cp5alTXK_ntnozRUIg7c4nWisdzVan0m-qgG8EqYGVFIRq0wcpIU39JsGB801KhB5IRxtRei9prF1hW6WE7FG1JGsJl-cM3vz-m1QahGMg Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: e7d1e420eec84d1f8b78333b7d814f3c Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesActionsTest-1853970953 Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:23:27.759765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/e7d1e420eec84d1f8b78333b7d814f3c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/e7d1e420eec84d1f8b78333b7d814f3c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/e7d1e420eec84d1f8b78333b7d814f3c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_e7d1e420eec84d1f8b78333b7d814f3c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesActionsTest-1853970953 Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: e7d1e420eec84d1f8b78333b7d814f3c Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesActionsTest-1853970953 Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesActionsTest-1853970953-project Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 9ee6eac2e6da4d83bd14324669dc7269 Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesActionsTest-1853970953-project Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 676959ea-2e7e-4410-aec0-645c6d2b078a could not be found. Oct 13 04:23:27.761140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '676959ea-2e7e-4410-aec0-645c6d2b078a'} Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 676959ea-2e7e-4410-aec0-645c6d2b078a could not be found. Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:23:27.761863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 676959ea-2e7e-4410-aec0-645c6d2b078a could not be found. Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 676959ea-2e7e-4410-aec0-645c6d2b078a could not be found. Oct 13 04:23:27.763137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:23:27.765388 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] Returning 404 to user: Instance 676959ea-2e7e-4410-aec0-645c6d2b078a could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:23:27.766403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-baef5aff-776a-4992-951d-20de870aa3f6 tempest-VolumesActionsTest-1853970953 tempest-VolumesActionsTest-1853970953-project] 10.222.0.27 "GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a" status: 404 len: 111 microversion: 2.1 time: 0.075483 Oct 13 04:23:27.786960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 196/352] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:27 2022] GET /compute/v2.1/servers/676959ea-2e7e-4410-aec0-645c6d2b078a => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:23:27.787892 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:27.787892 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:27.799988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:27.800492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:27.800928 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:27.801647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:27.802103 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:27.802555 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:27.898503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:27.898503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:27.903197 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:27.904264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:27.904849 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:27.905879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:27.906530 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:27.907188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:28.075316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:28.084832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-031403d2-5b2a-43f3-b678-01af4c90f7c2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.193134 Oct 13 04:23:28.086664 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 197/353] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:27 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:28.097063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:28.097861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:28.105616 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:28.106827 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:28.107889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:28.109062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:28.109763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:28.111681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:28.260297 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:28.270362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:28.272882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:28.274475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-643ee822-df42-4f82-87bc-076ad427820f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.186398 Oct 13 04:23:28.277592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 198/354] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:28 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:28.292776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-7d87b2e2-b902-465e-a149-01bbe72b422f req-70033eb1-40f9-4f96-baee-2d53f5046388 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "168caf0b-e6d8-441d-adfd-821ed4238c11", "tag": "4f0798be-0676-4fd9-984d-09cbaaf88866"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:28.306367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7d87b2e2-b902-465e-a149-01bbe72b422f req-70033eb1-40f9-4f96-baee-2d53f5046388 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:28.306977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7d87b2e2-b902-465e-a149-01bbe72b422f req-70033eb1-40f9-4f96-baee-2d53f5046388 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:28.307536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7d87b2e2-b902-465e-a149-01bbe72b422f req-70033eb1-40f9-4f96-baee-2d53f5046388 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:28.330191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-7d87b2e2-b902-465e-a149-01bbe72b422f req-70033eb1-40f9-4f96-baee-2d53f5046388 service nova] Creating event network-changed:4f0798be-0676-4fd9-984d-09cbaaf88866 for instance 168caf0b-e6d8-441d-adfd-821ed4238c11 on hv-cinder-86099 Oct 13 04:23:28.335603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-7d87b2e2-b902-465e-a149-01bbe72b422f req-70033eb1-40f9-4f96-baee-2d53f5046388 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056458 Oct 13 04:23:28.336445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 199/355] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:28.509202 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:28.520094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-75fe7f59-7572-416a-b89e-c13f13fccf4c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.743581 Oct 13 04:23:28.520929 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 157/356] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:27 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:29.109665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:29.111409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:29.118549 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:29.119122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:29.119751 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:29.120514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:29.121161 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:29.121764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:29.304673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:29.305367 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:29.312242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:29.313881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:29.314581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:29.315364 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:29.315888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:29.316618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:29.485495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:29.494093 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:29.495581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:29.496752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2bdc0ac4-b065-4ec9-b9ad-aa47801b3dbc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.196749 Oct 13 04:23:29.497315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 158/357] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:29 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:29.544997 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:29.545599 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:29.551727 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:29.552208 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:29.552634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:29.553319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:29.553763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:29.554181 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:29.569425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:29.577686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b4fc0ed6-4f2d-437d-9719-88c0fe815b8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.472826 Oct 13 04:23:29.578814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 200/358] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:29 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:29.693282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:29.702618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4df26783-980a-46dd-946f-041fe92bd1eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.161361 Oct 13 04:23:29.703276 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 159/359] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:29 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:30.370629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-2c9756c2-67a9-4dc0-9581-b684ecd6f01f req-0b25df24-8612-49ba-8f55-401f3dcafd1d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3720a7c-f5a9-468e-8195-221790e8931c", "tag": "fb3e21a4-2e2b-468a-8003-e487721c3939"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:30.376643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:30.383071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2c9756c2-67a9-4dc0-9581-b684ecd6f01f req-0b25df24-8612-49ba-8f55-401f3dcafd1d service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.383620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2c9756c2-67a9-4dc0-9581-b684ecd6f01f req-0b25df24-8612-49ba-8f55-401f3dcafd1d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.384140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2c9756c2-67a9-4dc0-9581-b684ecd6f01f req-0b25df24-8612-49ba-8f55-401f3dcafd1d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.399500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-2c9756c2-67a9-4dc0-9581-b684ecd6f01f req-0b25df24-8612-49ba-8f55-401f3dcafd1d service nova] Creating event network-changed:fb3e21a4-2e2b-468a-8003-e487721c3939 for instance f3720a7c-f5a9-468e-8195-221790e8931c on hv-cinder-86099 Oct 13 04:23:30.405124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-2c9756c2-67a9-4dc0-9581-b684ecd6f01f req-0b25df24-8612-49ba-8f55-401f3dcafd1d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038403 Oct 13 04:23:30.410255 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 201/360] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:30.524105 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:30.524826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:30.531286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.531843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.532359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.533121 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.533640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.534129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.607424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:30.608631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:30.617223 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:30.619204 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.620881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.621525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.622710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.623456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.624034 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.685288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:30.693744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:30.695216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:30.696370 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-dd4ee0ac-9075-4f05-adb8-f5afb6489450 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.176544 Oct 13 04:23:30.697241 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 160/361] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:30 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:30.727646 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:30.728637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:30.738880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.739781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.740284 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.741048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:30.741674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:30.742161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:30.832636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=28 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:30.876963 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:30.884779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-007b1377-7f98-43bb-9460-c0fb4be0a757 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.161724 Oct 13 04:23:30.887597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:30.894898 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 161/362] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:30 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:30.895576 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-36555964-f768-4e64-9843-80e89c4663b3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.297348 Oct 13 04:23:30.896627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 202/363] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:30 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:31.725355 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:31.725957 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:31.728345 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=76 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:31.733679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:31.734159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:31.734621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:31.735337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:31.735787 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:31.736212 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:31.740253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:31.867610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:31.876115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:31.877625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:31.878816 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f8b4d908-2aed-48cc-8dd4-a5b803d48d48 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.157489 Oct 13 04:23:31.879375 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 162/364] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:31 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:31.909833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:31.910465 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:31.917347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:31.917827 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:31.918285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:31.918970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:31.919155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:31.919590 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:31.919761 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:31.920392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:31.927152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:31.927640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:31.928074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:31.928822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:31.929282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:31.929703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:32.064244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:32.067338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:32.072254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cead7454-4bae-4304-99d3-d8be60c6ad83 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.168945 Oct 13 04:23:32.072814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 203/365] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:31 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:32.075153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0184df2d-a137-4d1b-9bc7-efed34442dbe tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.160554 Oct 13 04:23:32.075790 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 163/366] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:31 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:32.904676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:32.905277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:32.912075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:32.912664 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:32.912968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:32.913642 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:32.914085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:32.914613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:33.044061 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:33.052300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:33.053806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:33.054923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7d3674e0-9f92-4846-997e-1d5cf52001fb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.154300 Oct 13 04:23:33.055665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 204/367] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:32 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:33.095305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:33.096012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:33.097686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:33.098500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:33.102949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:33.103522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:33.104032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:33.104731 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:33.105183 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:33.105303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:33.105608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:33.105725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:33.106494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:33.107221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:33.107675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:33.108098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:33.253940 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:33.259501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:33.267594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-16785d9e-6639-4c5d-ab4f-ffb71cb5e727 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.176785 Oct 13 04:23:33.268542 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 164/368] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:33 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:33.273797 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c00a43ac-de3c-4b63-8905-5a99ed97f115 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.180824 Oct 13 04:23:33.274528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 205/369] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:33 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:33.660176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1917110525", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-243333657", "security_groups": [{"name": "tempest-secgroup-smoke-1367108023"}], "networks": [{"uuid": "f0b2fdc5-8dfa-4d54-8bae-0c0a1ba3fce3"}], "block_device_mapping_v2": [{"uuid": "738ff1ed-3ea3-4dd6-97e4-993700958e58", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:33.809584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] validate_networks() for [('f0b2fdc5-8dfa-4d54-8bae-0c0a1ba3fce3', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:23:34.080398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:34.081128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:34.095386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.097045 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.097723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.098866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.100030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.106808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.348454 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:34.371784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:34.374908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:34.377763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-90ea6035-4ec0-4f94-a214-6829bdedeafb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.301351 Oct 13 04:23:34.395215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 206/370] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:34 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:34.405593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:34.407948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:34.414291 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.415031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.415450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.416316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.417241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.417709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.624125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:34.627908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:34.633750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8cef1ca9-94c9-4769-8f4a-9fbe53152651 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.237007 Oct 13 04:23:34.635021 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 207/371] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:34 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:34.642121 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:34.643120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:34.649455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.650246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.650917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.651835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.652512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.653142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.869171 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:23:34.869698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:34.871549 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:23:34.872361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:23:34.872998 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:23:34.873822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:23:34.877963 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-15713ffc-1013-4ca9-864c-a80c1f83b3aa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.241035 Oct 13 04:23:34.878741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 208/372] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:34 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:34.897944 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.898763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.899376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.900277 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:34.900911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:34.901513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:34.930119 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Getting quotas for project fabffb6796a34773987a4f7e414352d0. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:23:34.948820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Getting quotas for user 242490d937324b31bc95ae1982ea0b39 and project fabffb6796a34773987a4f7e414352d0. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:23:35.041515 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:23:35.044160 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='738ff1ed-3ea3-4dd6-97e4-993700958e58',volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:23:35.317947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-988b5a90-25c7-44f8-a6e7-2b4568936d21 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.662287 Oct 13 04:23:35.319200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 165/373] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:23:33 2022] POST /compute/v2.1/servers => generated 406 bytes in 1664 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 13 04:23:35.331011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-04d830f2-1ff6-497a-b97f-8670d34c1f5f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:35.331715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-04d830f2-1ff6-497a-b97f-8670d34c1f5f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:35.427743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:35.428552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:35.441909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.445001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.445001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:35.446492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-04d830f2-1ff6-497a-b97f-8670d34c1f5f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:35.452265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.455241 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.456015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-04d830f2-1ff6-497a-b97f-8670d34c1f5f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1088 microversion: 2.1 time: 0.127600 Oct 13 04:23:35.466677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:35.466787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 209/374] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:35 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1088 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:35.480813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ab91ae1f-8df6-40b8-8892-f870059f692c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:35.489202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ab91ae1f-8df6-40b8-8892-f870059f692c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.490090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ab91ae1f-8df6-40b8-8892-f870059f692c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.490719 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ab91ae1f-8df6-40b8-8892-f870059f692c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:35.604289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ab91ae1f-8df6-40b8-8892-f870059f692c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Going to try to stop instance {{(pid=118673) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 13 04:23:35.760959 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ab91ae1f-8df6-40b8-8892-f870059f692c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action" status: 202 len: 0 microversion: 2.1 time: 0.267528 Oct 13 04:23:35.769961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 210/375] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:23:35 2022] POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action => generated 0 bytes in 294 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:23:35.796658 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:35.797443 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:35.816317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.816822 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.817268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:35.817988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.818471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.818899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:35.828394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:35.849388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:35.868066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:35.870603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fccfc7d6-ac22-49ff-b78b-42677b789829 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1744 microversion: 2.20 time: 0.449839 Oct 13 04:23:35.872319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 166/376] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:35 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1744 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:35.880560 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:35.881507 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:35.933011 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.934638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.936148 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:35.939079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:35.940006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:35.940736 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:36.119103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:36.128514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4d5c4f24-42e7-4194-afb3-95731e826157 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.356998 Oct 13 04:23:36.129527 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 211/377] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:35 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:36.134036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:36.134674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:36.141593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:36.142138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:36.142750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:36.143465 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:36.143981 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:36.144398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:36.257869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:36.270353 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d7210ba7-3a01-41bb-8443-0d22ad4c1cc3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.396367 Oct 13 04:23:36.284048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 167/378] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:35 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:36.325250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:36.335691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6f39a721-8376-445b-a876-17aeafe80b28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.204995 Oct 13 04:23:36.338344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 212/379] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:36 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:36.483135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:36.483982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:36.495460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:36.496232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:36.496966 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:36.497844 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:36.498503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:36.499083 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:36.596728 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:36.605071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8b5369e9-ac53-40c8-9f29-5c31b6fe03d4 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1138 microversion: 2.1 time: 0.126717 Oct 13 04:23:36.606637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 168/380] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:36 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1138 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:36.897151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:36.897819 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:36.905656 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:36.906399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:36.906918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:36.907885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:36.908734 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:36.909362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.089206 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.107913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.109725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.111925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3a886c3c-e884-4640-8f8a-04c83fea7446 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1744 microversion: 2.20 time: 0.218639 Oct 13 04:23:37.126550 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 213/381] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:36 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1744 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:37.153545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:37.154270 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:37.167370 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.167866 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.168305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.169018 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.169473 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.173317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.295582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:37.296392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:37.303306 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.303950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.304640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.305482 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.306058 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.307310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.519978 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.532386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0f543701-cb2a-47f1-9654-b24330b368c6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.240657 Oct 13 04:23:37.533488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 214/382] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:37 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:37.558142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:37.558780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:37.570727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.571517 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.572113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.575197 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.575859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.576442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.639347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.654079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1ee9f9ab-3d42-463d-947d-0ce0cd62b3eb tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.504483 Oct 13 04:23:37.654972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 169/383] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:37 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:37.663558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:37.664410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:37.675466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.676280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.676917 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.678658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:37.679151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:37.679617 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:37.755760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.767580 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c2157b15-a5d2-4694-8810-3c7bf9a0f103 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1138 microversion: 2.1 time: 0.110175 Oct 13 04:23:37.768828 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 170/384] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:37 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1138 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:37.785825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:37.794647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4d3eabd9-441e-4b64-862f-34243b1c6b2e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.259751 Oct 13 04:23:37.795753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 215/385] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:37 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:38.140763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:38.142403 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:38.148976 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.149470 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.149909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.150641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.151098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.151519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.379285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:38.395714 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:38.398144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:38.400481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8e87f5db-f33b-4ce9-b93f-edfb0ec6f8a2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.264137 Oct 13 04:23:38.402115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 171/386] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:38 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:38.572287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:38.573122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:38.589390 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.589879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.590345 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.597323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.597882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.598366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.680304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:38.681008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:38.693720 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.696014 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.696888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.701506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.702264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.702817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.855125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:38.866750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d8e6b7fc-fe67-4b58-804e-678b0d2b79c9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.311825 Oct 13 04:23:38.868817 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 216/387] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:38 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:38.880016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:38.880939 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:38.904487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.905293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.909531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.911031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:38.911763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:38.912425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:38.997219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:39.006168 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1e32fe72-4a54-4cd3-b7c0-0609f5f2d5ce tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.332125 Oct 13 04:23:39.022231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 172/388] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:38 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:39.022231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:39.022231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:39.037570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.038063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.038523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:39.042547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.043040 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.044856 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:39.050890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:39.068774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f755fe29-c32b-46a6-9d85-40358a2dca16 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1186 microversion: 2.1 time: 0.196905 Oct 13 04:23:39.072933 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 217/389] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:38 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1186 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:39.277271 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-f98af08d-a704-4ad8-9fb7-7c5c38697acc req-60899f7d-5d16-4a87-9855-7b5f8a4a70f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "168caf0b-e6d8-441d-adfd-821ed4238c11", "name": "network-vif-plugged", "status": "completed", "tag": "4f0798be-0676-4fd9-984d-09cbaaf88866"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:39.309626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-f98af08d-a704-4ad8-9fb7-7c5c38697acc req-60899f7d-5d16-4a87-9855-7b5f8a4a70f3 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.310106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-f98af08d-a704-4ad8-9fb7-7c5c38697acc req-60899f7d-5d16-4a87-9855-7b5f8a4a70f3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.310583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-f98af08d-a704-4ad8-9fb7-7c5c38697acc req-60899f7d-5d16-4a87-9855-7b5f8a4a70f3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:39.373982 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-f98af08d-a704-4ad8-9fb7-7c5c38697acc req-60899f7d-5d16-4a87-9855-7b5f8a4a70f3 service nova] Creating event network-vif-plugged:4f0798be-0676-4fd9-984d-09cbaaf88866 for instance 168caf0b-e6d8-441d-adfd-821ed4238c11 on hv-cinder-86099 Oct 13 04:23:39.400096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-f98af08d-a704-4ad8-9fb7-7c5c38697acc req-60899f7d-5d16-4a87-9855-7b5f8a4a70f3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.125551 Oct 13 04:23:39.401327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 218/390] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:39.425302 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:39.436084 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:39.436764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:39.442184 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-75eee4d7-f902-43f0-ba4e-8c425539bd09 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.430601 Oct 13 04:23:39.444749 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 173/391] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:39 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:39.472865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.473728 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.474377 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:39.476079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.478192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.482786 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:39.808890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:39.821720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:39.824363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:39.826395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0f6e210d-6286-474c-9dab-078e83e9a704 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.394574 Oct 13 04:23:39.847037 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 219/392] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:39 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:39.896016 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:39.898743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:39.912836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.914114 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.914680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:39.915402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:39.915860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:39.916855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.043432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:40.046801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:40.057859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.058362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.058790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.059484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.059928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.075221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.382388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:40.399827 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:40.408328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e427370f-6dfb-47a2-9239-57c5aba0511d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.516887 Oct 13 04:23:40.425308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 174/393] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:39 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:40.431457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c828be06-2bb0-4856-b1ca-0a77422b9f0a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.402039 Oct 13 04:23:40.442529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 220/394] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:40 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:40.449215 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:40.450259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:40.467793 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.468454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.469746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.472877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.473398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.473861 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.483787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:40.484579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:40.496054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.496545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.496977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.497755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.498474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.499072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.604139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:40.619714 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3b774cf8-6897-4d4f-92db-fa0674ae9ffd tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1204 microversion: 2.1 time: 0.191813 Oct 13 04:23:40.621116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 175/395] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:40 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1204 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:40.795809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:40.815569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-df618a35-ec83-43fc-be5c-10951e0fc5fa tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.339999 Oct 13 04:23:40.820433 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 221/396] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:40 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:40.858019 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:40.858980 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:40.880690 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.881523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.883433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:40.886651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:40.887352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:40.889150 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.444254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:41.445735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:41.475795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.476719 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.477385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.484890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.485675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.489372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.772349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:41.785366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-36e49650-de5b-4242-a9ec-af8a590b6857 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.356377 Oct 13 04:23:41.786685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 222/397] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:41 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:41.794509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-43282552-28fb-44f8-a8f3-5e781fe2316d req-17672066-7987-442a-9c35-ffa505432cf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "name": "network-vif-plugged", "status": "completed", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:41.807869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:41.810448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-43282552-28fb-44f8-a8f3-5e781fe2316d req-17672066-7987-442a-9c35-ffa505432cf9 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.811873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-43282552-28fb-44f8-a8f3-5e781fe2316d req-17672066-7987-442a-9c35-ffa505432cf9 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.812534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-43282552-28fb-44f8-a8f3-5e781fe2316d req-17672066-7987-442a-9c35-ffa505432cf9 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.821358 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:41.825151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:41.831751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aa173dd3-12d2-4626-9712-04e4a45d35a1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.979430 Oct 13 04:23:41.834193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 176/398] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:40 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:41.842764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-43282552-28fb-44f8-a8f3-5e781fe2316d req-17672066-7987-442a-9c35-ffa505432cf9 service nova] Creating event network-vif-plugged:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:23:41.850805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:41.852208 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:41.858512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-43282552-28fb-44f8-a8f3-5e781fe2316d req-17672066-7987-442a-9c35-ffa505432cf9 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063982 Oct 13 04:23:41.859428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 223/399] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:41.861337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.862251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.862842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.863757 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.864448 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.865038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.872104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:41.877000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:41.887495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.888385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.889078 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:41.890122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:41.891421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:41.892247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:42.017683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:42.034806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-35f73a2d-62f6-482c-92c5-d2acd90ee492 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1192 microversion: 2.1 time: 0.172367 Oct 13 04:23:42.037204 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 224/400] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:41 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1192 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:42.044700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:42.045521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:42.053303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:42.054140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:42.054839 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:42.055667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:42.056328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:42.056871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:42.316952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:42.330180 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b999dc16-3fe2-413d-b6ff-55813dbe33b3 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1372 microversion: 2.1 time: 0.491551 Oct 13 04:23:42.332863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 177/401] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:41 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1372 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:42.401064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:42.404980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:42.419175 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a9c306f1-4159-47f3-b86c-87c73bee58be tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.379541 Oct 13 04:23:42.437566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 225/402] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:42 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:42.812981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:42.813984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:42.843788 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:42.848213 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:42.849098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:42.850503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:42.851150 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:42.851812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:42.874558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:42.876367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:42.904938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:42.908025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:42.909014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:42.918553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:42.919901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:42.920988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:43.356969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:43.368814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-49bb9446-b7f5-4fc0-8b24-b7ea94191805 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.559755 Oct 13 04:23:43.369412 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 178/403] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:42 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:43.387708 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:43.389323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:43.390129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:43.397380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:43.398880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:43.402924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:43.403394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:43.405681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:43.409195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:43.409858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:43.410550 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:43.410976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-deb94f67-0c30-4a3f-8ba2-9c47efc6fd88 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.531811 Oct 13 04:23:43.417440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 226/404] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:42 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:43.424075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:43.424692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:43.456848 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:43.463694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:43.463997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:43.464689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:43.465296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:43.465632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:43.466063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:43.777385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:43.797888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0196960e-6238-4b22-a8e4-9ca60116b8ca tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1566 microversion: 2.1 time: 0.378555 Oct 13 04:23:43.806620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 227/405] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:43 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1566 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:43.828829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:43.829494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:43.849327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:43.849864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:43.850437 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:43.855514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:43.856003 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:43.856775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.069886 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:44.079616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-37e6382b-9a99-4769-8159-fd7abf505fc9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.698253 Oct 13 04:23:44.084601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 179/406] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:43 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:44.091742 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:44.092233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:44.102855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.103456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.103965 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.104700 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.105172 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.105658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.228373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:44.238353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-30b4304a-1c79-4868-9fcb-511a7802c4e8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.413576 Oct 13 04:23:44.250471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 228/407] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:43 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:44.250471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-ae2b7ae7-46df-4310-9987-b5323a8ec484 req-4a8c9cb3-84ba-435d-aa54-7327b606641a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3720a7c-f5a9-468e-8195-221790e8931c", "name": "network-vif-plugged", "status": "completed", "tag": "fb3e21a4-2e2b-468a-8003-e487721c3939"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:44.259636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-ae2b7ae7-46df-4310-9987-b5323a8ec484 req-4a8c9cb3-84ba-435d-aa54-7327b606641a service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.260346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-ae2b7ae7-46df-4310-9987-b5323a8ec484 req-4a8c9cb3-84ba-435d-aa54-7327b606641a service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.260775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-ae2b7ae7-46df-4310-9987-b5323a8ec484 req-4a8c9cb3-84ba-435d-aa54-7327b606641a service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.273936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-ae2b7ae7-46df-4310-9987-b5323a8ec484 req-4a8c9cb3-84ba-435d-aa54-7327b606641a service nova] Creating event network-vif-plugged:fb3e21a4-2e2b-468a-8003-e487721c3939 for instance f3720a7c-f5a9-468e-8195-221790e8931c on hv-cinder-86099 Oct 13 04:23:44.279225 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-ae2b7ae7-46df-4310-9987-b5323a8ec484 req-4a8c9cb3-84ba-435d-aa54-7327b606641a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034428 Oct 13 04:23:44.281107 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 229/408] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:44.285347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-28e45d27-2c5d-4daa-b448-343648147538 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.199348 Oct 13 04:23:44.289553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 180/409] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:23:44 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-interface => generated 284 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:44.400862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:44.405894 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:44.427310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.429050 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.433294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.434383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.435056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.435725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.451635 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:44.452633 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:44.466400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.471699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.472455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.476435 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:44.481683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:44.488592 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:44.909952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:44.913715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:44.931015 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:44.932723 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:44.938800 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c6ea2ff1-bdb6-4356-af6b-666ce5f6b991 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.491949 Oct 13 04:23:44.940991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 181/410] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:44 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:44.979338 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:44.998372 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b88ac5d8-cd44-42d5-ab4e-08efaadaaefb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.603335 Oct 13 04:23:45.023686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 230/411] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:44 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:45.107231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:45.109414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:45.123689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:45.127196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.128033 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.129242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:45.130387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.131347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.143679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:45.187335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-554e5047-1d80-4f6a-9ecd-7e6586f7921a req-fe60cbf4-7ca8-452e-803e-5ceffad99b13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:45.209117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-554e5047-1d80-4f6a-9ecd-7e6586f7921a req-fe60cbf4-7ca8-452e-803e-5ceffad99b13 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.212299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-554e5047-1d80-4f6a-9ecd-7e6586f7921a req-fe60cbf4-7ca8-452e-803e-5ceffad99b13 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.212735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-554e5047-1d80-4f6a-9ecd-7e6586f7921a req-fe60cbf4-7ca8-452e-803e-5ceffad99b13 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:45.238631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-554e5047-1d80-4f6a-9ecd-7e6586f7921a req-fe60cbf4-7ca8-452e-803e-5ceffad99b13 service nova] Creating event network-changed:9b98fb07-be4c-4add-b232-a8149b2588d6 for instance f186151e-3b9a-4cdf-8d6e-427fea09de90 on hv-cinder-86099 Oct 13 04:23:45.257287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-554e5047-1d80-4f6a-9ecd-7e6586f7921a req-fe60cbf4-7ca8-452e-803e-5ceffad99b13 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071585 Oct 13 04:23:45.258569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 231/412] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:45.271785 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:45.273965 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:45.287899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.292331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.294003 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:45.298689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.299352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.302053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:45.450961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:45.476551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:45.487600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-60583cd7-d3a5-45f7-89e5-38684fbc84be tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.385752 Oct 13 04:23:45.488860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 182/413] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:45 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:45.804072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:45.824655 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5a35d1de-300b-49a6-a5b6-5ba5b7134fb0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1366 microversion: 2.1 time: 0.564427 Oct 13 04:23:45.838647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 232/414] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:45 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1366 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:45.965269 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:45.966438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:45.988067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.988566 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.989209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:45.989974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:45.990443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:45.990866 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.033279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:46.034597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:46.052297 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.053654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.057163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.058798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.062268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.063889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.528683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:46.538668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ac2e7154-101b-47d0-a2be-d4cafd2df13b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.510806 Oct 13 04:23:46.554578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 233/415] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:46 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:46.564152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:46.564858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:46.577542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.600996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:46.610540 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.028s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.611243 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.612337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.612878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.613374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:46.613885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.615522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:46.623629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a5467e59-2639-4457-87e8-a73f7fe0cbf0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.662283 Oct 13 04:23:46.630539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 183/416] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:45 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:46.685140 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-1983dbc4-b4da-40ba-ae2f-f26997880410 req-89baaacc-e187-4f6c-bf25-ad86579d32f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "168caf0b-e6d8-441d-adfd-821ed4238c11", "tag": "4f0798be-0676-4fd9-984d-09cbaaf88866"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:46.714953 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1983dbc4-b4da-40ba-ae2f-f26997880410 req-89baaacc-e187-4f6c-bf25-ad86579d32f6 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.716878 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1983dbc4-b4da-40ba-ae2f-f26997880410 req-89baaacc-e187-4f6c-bf25-ad86579d32f6 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.719039 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1983dbc4-b4da-40ba-ae2f-f26997880410 req-89baaacc-e187-4f6c-bf25-ad86579d32f6 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.744495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-1983dbc4-b4da-40ba-ae2f-f26997880410 req-89baaacc-e187-4f6c-bf25-ad86579d32f6 service nova] Creating event network-changed:4f0798be-0676-4fd9-984d-09cbaaf88866 for instance 168caf0b-e6d8-441d-adfd-821ed4238c11 on hv-cinder-86099 Oct 13 04:23:46.761499 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-1983dbc4-b4da-40ba-ae2f-f26997880410 req-89baaacc-e187-4f6c-bf25-ad86579d32f6 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080032 Oct 13 04:23:46.762127 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 184/417] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:23:46.861991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:46.864600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:46.886636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.896735 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.897565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:46.898390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:46.898873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:46.899319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:47.034519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:47.049764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1935c711-8b13-4508-a813-a28b768af2fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.493828 Oct 13 04:23:47.058923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 234/418] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:46 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:47.585257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:47.586511 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:47.598798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:47.608100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:47.612005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:47.613301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:47.613947 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:47.614569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:47.759626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:47.779065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6a8180ac-d21f-41a6-9ada-0eb854cc1c1a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1556 microversion: 2.1 time: 0.921580 Oct 13 04:23:47.788055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 185/419] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:46 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1556 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:47.813320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:47.827800 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:47.860775 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:47.863805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:47.866747 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:47.868748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:47.875017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:47.875747 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:47.948096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:47.961075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-142f663d-6510-4de5-a502-d2d5143525ac tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.384518 Oct 13 04:23:47.979239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 235/420] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:47 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:47.994281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:48.007001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:48.019608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:48.020392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:48.020997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:48.021871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:48.022499 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:48.023076 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:48.210805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-91ca4bdc-057d-42f1-bf93-563e62841a92 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.229629 Oct 13 04:23:48.211382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 236/421] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:23:47 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/os-interface => generated 284 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:48.216214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:48.217010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:48.243981 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:48.244710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:48.245338 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:48.246236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:48.249198 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:48.249844 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:48.534677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:48.544544 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:48.546003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:48.565308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-72fef735-9d3a-456c-b05b-e836f7efc0ec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.757895 Oct 13 04:23:48.568379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 186/422] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:47 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:49.013096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:49.014608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:49.028982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:49.029948 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:49.032310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:49.033330 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:49.034828 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:49.036053 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:49.315980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:49.324388 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1ae7d102-ca43-4eab-9b49-c1ff978d9559 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 1.111658 Oct 13 04:23:49.324976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 237/423] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:48 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 1113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:49.564920 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:49.597646 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f7bbbea1-b2b7-46f7-8b98-9274afc6b658 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.594055 Oct 13 04:23:49.604027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:49.608457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:49.616622 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 187/424] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:49 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:49.654826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:49.664629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:49.666929 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:49.677955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:49.684276 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:49.685948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:50.224630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:50.263456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:50.269047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:50.278022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c11df230-edd0-4a44-b363-9e0394f319e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.675921 Oct 13 04:23:50.281116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 238/425] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:49 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:50.349031 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:50.349637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:50.363521 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:50.367213 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:50.368280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:50.373821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:50.375232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:50.378003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:50.639570 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:50.640172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:50.658631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:50.661781 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:50.670159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:50.671717 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:50.673995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:50.675714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:50.825005 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:50.837805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e84ce0be-6be7-4e85-b608-60c6c00550f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.489787 Oct 13 04:23:50.851091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 188/426] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:50 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:51.055287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:51.070775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-debd2c86-f350-47da-81f2-1ea7fb3ca355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.438953 Oct 13 04:23:51.075684 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 239/427] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:50 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:51.308220 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:51.308995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:51.332482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:51.336263 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:51.339937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:51.341875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:51.344745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:51.349161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:51.780304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-7f156e61-04b0-4423-a632-164c62e7abfd req-82e476a7-e951-438b-abe2-95f9e9c9e558 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3720a7c-f5a9-468e-8195-221790e8931c", "tag": "fb3e21a4-2e2b-468a-8003-e487721c3939"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:51.797486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:51.809597 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:51.828239 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:51.830673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7f156e61-04b0-4423-a632-164c62e7abfd req-82e476a7-e951-438b-abe2-95f9e9c9e558 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:51.831163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7f156e61-04b0-4423-a632-164c62e7abfd req-82e476a7-e951-438b-abe2-95f9e9c9e558 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:51.831664 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7f156e61-04b0-4423-a632-164c62e7abfd req-82e476a7-e951-438b-abe2-95f9e9c9e558 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:51.835724 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d049cf80-7e70-4f2c-8328-4dbaa14f0ed1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.533944 Oct 13 04:23:51.851604 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 189/428] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:51 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:51.874261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:51.875839 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:51.903434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:51.907103 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:51.908101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:51.912766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-7f156e61-04b0-4423-a632-164c62e7abfd req-82e476a7-e951-438b-abe2-95f9e9c9e558 service nova] Creating event network-changed:fb3e21a4-2e2b-468a-8003-e487721c3939 for instance f3720a7c-f5a9-468e-8195-221790e8931c on hv-cinder-86099 Oct 13 04:23:51.922043 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:51.923607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:51.924506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:51.934223 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-7f156e61-04b0-4423-a632-164c62e7abfd req-82e476a7-e951-438b-abe2-95f9e9c9e558 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.160559 Oct 13 04:23:51.937542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 240/429] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:23:52.110986 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:52.111942 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:52.125759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:52.129191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:52.129847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:52.130803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:52.131421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:52.132952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:52.533052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:52.544025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-82a76e71-8c9b-4725-9b03-b3ac646b64e6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.674571 Oct 13 04:23:52.555152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 190/430] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:51 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:52.580960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:52.599235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2021878f-0594-435d-a8d9-4b628419c035 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.493269 Oct 13 04:23:52.599884 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 241/431] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:52 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:52.887021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:52.887893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:52.900398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:52.901089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:52.901703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:52.902762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:52.903552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:52.904146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:53.152351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:53.167317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:53.176978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:53.178528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-54e657b0-6c65-4fe6-81f0-7579a649b79c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.295539 Oct 13 04:23:53.204904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 191/432] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:52 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:53.586112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:53.587286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:53.607700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:53.608612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:53.609337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:53.610314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:53.610923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:53.611515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:53.635495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:53.637506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:53.649536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:53.651253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:53.652242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:53.653295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:53.654017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:53.654987 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:54.107326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:54.119346 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-27885988-a9a4-4a8e-8109-7ba82fa21139 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.486340 Oct 13 04:23:54.124420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 192/433] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:53 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:54.212451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:54.214771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:54.232041 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:54.232825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:54.233411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:54.234464 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:54.235192 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:54.242146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:54.366409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:54.374118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-58aaab24-36b3-4ec2-a6ee-c8eeb3e5dc0b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.792038 Oct 13 04:23:54.383228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 242/434] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:53 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:54.541223 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:54.557624 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:54.559565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:54.566051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-46c8ee02-c3c6-4cba-97d0-6413b93d9d42 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.357596 Oct 13 04:23:54.569222 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 193/435] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:54 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:55.152642 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:55.155700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:55.190787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:55.191511 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:55.191980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:55.193001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:55.193720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:55.194256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:55.428823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:55.429554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:55.457567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:55.459536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:55.464013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:55.465169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:55.469728 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:55.470774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:55.522082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:55.543645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-143f987c-2798-4880-b25e-7a62241d1de7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.395984 Oct 13 04:23:55.568299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 243/436] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:55 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:55.602827 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:55.604318 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:55.630993 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:55.632144 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:55.632805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:55.635134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:55.641479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:55.644027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:55.842238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:55.877192 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d353e78c-b4fa-4eed-90f4-4154cf57c7b5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.475565 Oct 13 04:23:55.879382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 194/437] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:55 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:23:55.985919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:55.990147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:55.996283 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:55.997774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:55.999062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-bc2ea86a-32ce-47a5-b32d-f81bd46b0c20 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.403886 Oct 13 04:23:55.999710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 244/438] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:55 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:23:56.575768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:56.576360 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:56.588651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:56.589149 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:56.590622 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:56.591557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:56.592603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:56.593059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:56.826309 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:56.838306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-eebe150d-8765-46d8-8082-7e3ca681bdb5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.267004 Oct 13 04:23:56.865997 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 195/439] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:56 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:56.911317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:56.912503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:56.922049 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:56.922575 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:56.923216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:56.924080 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:56.924682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:56.925252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:57.028151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:57.029054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:57.043834 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:57.048582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:57.049241 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:57.052812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:57.056223 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:57.056669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:57.141639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:57.150693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7895b51e-15fd-4ba2-8410-1bb2a4e108bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.243221 Oct 13 04:23:57.157916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 245/440] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:56 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:57.279206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:57.292849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:57.296398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:57.298678 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-099e8d99-9cc5-49c8-aa08-bfe116b4aab4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1739 microversion: 2.20 time: 0.275727 Oct 13 04:23:57.299863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 196/441] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:57 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1739 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:57.870624 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:57.871908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:57.879041 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:57.880267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:57.881089 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:57.882281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:57.882953 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:57.883587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:58.084455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:58.095129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3f92d8dd-8d6d-49e3-a35b-e25433933ab0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.228509 Oct 13 04:23:58.114639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 246/442] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:57 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:58.185711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:58.189679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:58.250865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:58.253728 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:58.254232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:58.255954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:58.257843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:58.258307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:58.272190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:23:58.331886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:58.332558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:58.347645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:58.348169 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:58.348614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:58.359551 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:58.360424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:58.361029 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:58.552423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:58.560705 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2a10462f-a673-4b29-9eb0-7f4a7324642d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.381703 Oct 13 04:23:58.561455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 197/443] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:58 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:58.605419 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:58.615587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:58.622533 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:58.624025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e90c17ff-2fb8-4846-9f18-42b233ad3639 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1721 microversion: 2.20 time: 0.296432 Oct 13 04:23:58.627445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 247/444] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:58 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1721 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:23:59.121047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:59.121957 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:59.135047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.135554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.136003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.136730 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.137197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.137638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.301410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:59.313509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bb493948-1669-4cf7-8625-69f7caaf6240 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.192832 Oct 13 04:23:59.314117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 198/445] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:59 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:59.585059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:59.585737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:59.595168 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.595694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.596139 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.596863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.597319 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.597745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.656741 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:23:59.660593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:23:59.668793 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.669918 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.673489 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.674789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.675465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.676098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.801317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:59.813557 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c59631d5-4d1a-43a5-95e1-d4313707fc7e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.232037 Oct 13 04:23:59.814625 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 248/446] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:23:59 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:23:59.845388 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-94013935-5ab5-482c-abda-ce09f2f87a45 req-f74f5ce3-f56e-44dc-b85c-c0aa6474f15c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "name": "network-vif-plugged", "status": "completed", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:23:59.858338 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-94013935-5ab5-482c-abda-ce09f2f87a45 req-f74f5ce3-f56e-44dc-b85c-c0aa6474f15c service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:23:59.859639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-94013935-5ab5-482c-abda-ce09f2f87a45 req-f74f5ce3-f56e-44dc-b85c-c0aa6474f15c service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:23:59.860289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-94013935-5ab5-482c-abda-ce09f2f87a45 req-f74f5ce3-f56e-44dc-b85c-c0aa6474f15c service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:23:59.875461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-94013935-5ab5-482c-abda-ce09f2f87a45 req-f74f5ce3-f56e-44dc-b85c-c0aa6474f15c service nova] Creating event network-vif-plugged:9b98fb07-be4c-4add-b232-a8149b2588d6 for instance f186151e-3b9a-4cdf-8d6e-427fea09de90 on hv-cinder-86099 Oct 13 04:23:59.881314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-94013935-5ab5-482c-abda-ce09f2f87a45 req-f74f5ce3-f56e-44dc-b85c-c0aa6474f15c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040092 Oct 13 04:23:59.882522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 249/447] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:23:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:23:59.942057 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:59.950348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:59.952013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:23:59.953927 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e3fd7bf1-5d9e-49fe-a054-f417770571f3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1721 microversion: 2.20 time: 0.302832 Oct 13 04:23:59.955585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 199/448] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:23:59 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1721 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:00.337434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:00.338123 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:00.344364 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:00.345062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:00.345498 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:00.346195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:00.346674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:00.347096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:00.500327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:00.509638 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c50d26a7-6799-41b9-b631-57edf8007b09 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.176441 Oct 13 04:24:00.510235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 250/449] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:00 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:00.841751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:00.842667 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:00.849775 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:00.852820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:00.853209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:00.853921 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:00.854410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:00.854832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:00.982602 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:00.983209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:00.986897 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:00.996649 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:00.997481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:00.998268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:00.999106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:01.000438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:01.000869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:01.056595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:01.068051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4c5070d4-b0df-45cd-8006-776a41e68dd3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.229745 Oct 13 04:24:01.069220 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 200/450] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:00 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:01.254469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:01.262552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:01.264041 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:01.265820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fd693d29-cdef-431a-9b4c-7d6c1ccf1888 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1721 microversion: 2.20 time: 0.287800 Oct 13 04:24:01.266542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 251/451] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:00 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1721 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:01.534713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:01.535565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:01.543323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:01.544034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:01.544649 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:01.545512 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:01.546112 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:01.546951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:01.689093 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:01.697623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-54fee53e-cc32-47f4-95b5-bbf8729c673e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.167970 Oct 13 04:24:01.698214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 201/452] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:01 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:01.899591 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-fc45cb13-e00a-4027-a4ac-291a0121d229 req-ff53fc58-335a-40f6-bdc4-b5a6b6f5a45d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "name": "network-vif-unplugged", "status": "completed", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:01.909396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-fc45cb13-e00a-4027-a4ac-291a0121d229 req-ff53fc58-335a-40f6-bdc4-b5a6b6f5a45d service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:01.910030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-fc45cb13-e00a-4027-a4ac-291a0121d229 req-ff53fc58-335a-40f6-bdc4-b5a6b6f5a45d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:01.910473 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-fc45cb13-e00a-4027-a4ac-291a0121d229 req-ff53fc58-335a-40f6-bdc4-b5a6b6f5a45d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:01.921552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-fc45cb13-e00a-4027-a4ac-291a0121d229 req-ff53fc58-335a-40f6-bdc4-b5a6b6f5a45d service nova] Creating event network-vif-unplugged:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:24:01.927688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-fc45cb13-e00a-4027-a4ac-291a0121d229 req-ff53fc58-335a-40f6-bdc4-b5a6b6f5a45d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031716 Oct 13 04:24:01.928780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 252/453] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:01 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:02.095996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:02.096621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:02.098954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:02.104170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.104659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.105090 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.105780 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.106525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.106953 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.245100 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:02.254167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0099d77f-58af-48f0-9544-4878dc677ac3 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.162000 Oct 13 04:24:02.257384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 202/454] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:02 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:02.293100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:02.294621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:02.301620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.302104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.302564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.303577 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.303703 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.304298 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.548498 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:02.557753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:02.559660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:02.561253 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4a144029-db54-41e0-9168-fee34280ef2b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1724 microversion: 2.20 time: 0.273345 Oct 13 04:24:02.562457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 253/455] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:02 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1724 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:02.588038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "53d91359-d388-47e5-8291-83967253236f"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:02.590481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:02.601665 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.602469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.603217 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.603915 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.604520 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.604941 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.721127 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:02.721762 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:02.728408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.729000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.729507 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.730479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:02.731032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:02.731919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:02.925433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-436f1bb2-f7bb-4c27-b5f1-f35e741d6762 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.341609 Oct 13 04:24:02.926248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 203/456] 10.222.0.27 () {62 vars in 1364 bytes} [Thu Oct 13 04:24:02 2022] POST /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments => generated 188 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:24:03.057050 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:03.057835 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:03.065024 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:03.065521 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:03.065960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:03.066693 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:03.067150 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:03.067613 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:03.111797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118674) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 13 04:24:03.114958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:03.124048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c2a01a0e-943d-44d6-9dfa-272248ae2275 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.406874 Oct 13 04:24:03.125326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 254/457] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:02 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:03.199774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-326668d8-5105-45fc-96c1-2d8ce84e6309 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/action" status: 202 len: 0 microversion: 2.20 time: 0.146705 Oct 13 04:24:03.206824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 204/458] 10.222.0.27 () {62 vars in 1319 bytes} [Thu Oct 13 04:24:03 2022] POST /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/action => generated 0 bytes in 148 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 13 04:24:03.212943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:03.213556 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:03.232173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:03.232735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:03.236847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:03.237778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:03.238278 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:03.238707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:03.277379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:03.277991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:03.289051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:03.289562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:03.290001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:03.290730 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:03.291391 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:03.292150 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:03.395393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:03.405229 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:03.422761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:03.423946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-de54e86a-cabc-4818-9c46-4a07a0e58d6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1810 microversion: 2.20 time: 0.217664 Oct 13 04:24:03.424824 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 255/459] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:03 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1810 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:03.496041 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:03.504670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-98b03049-af3b-4083-aa08-ca1a9ba5d154 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.231309 Oct 13 04:24:03.505337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 205/460] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:03 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:04.148682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:04.149807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:04.159206 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:04.159780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:04.160316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:04.161352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:04.161593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:04.162120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:04.313712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:04.326469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8f043dab-713e-4af9-a343-780d51248492 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.181844 Oct 13 04:24:04.328002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 256/461] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:04 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:04.454757 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:04.456437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:04.463707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:04.464189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:04.464879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:04.465768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:04.466405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:04.467022 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:04.530361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:04.533885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:04.542152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:04.542688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:04.543131 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:04.543839 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:04.544294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:04.544900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:04.619100 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:04.633179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:04.635578 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:04.636954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-86bdcccf-4f41-4e0b-b589-cc2c38c947c9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1810 microversion: 2.20 time: 0.189780 Oct 13 04:24:04.637693 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 206/462] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:04 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1810 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:04.749490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:04.757802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-38ed49a4-6fbf-459a-a3df-b2b73966f1bc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1260 microversion: 2.1 time: 0.233614 Oct 13 04:24:04.758932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 257/463] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:04 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1260 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:05.350174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:05.351097 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:05.357359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:05.357836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:05.358283 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:05.359532 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:05.360041 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:05.360453 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:05.506597 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:05.519929 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-474a9e86-2e17-4835-bb6f-238d8992169d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.172926 Oct 13 04:24:05.520525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 207/464] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:05 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:05.663945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:05.666079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:05.676145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:05.677018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:05.677732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:05.678704 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:05.679379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:05.680186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:05.784038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:05.785162 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:05.791836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:05.793518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:05.794420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:05.795390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:05.796473 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:05.797747 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:05.973891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:05.982991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6b34b0eb-da4d-481f-8b13-ce7e4be776f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1445 microversion: 2.1 time: 0.203772 Oct 13 04:24:05.985114 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 208/465] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:05 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1445 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:05.992644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:06.007226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:06.007870 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:06.008697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:06.009549 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:06.011216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a5d746e9-b846-422f-b3cb-384cabeb796f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1810 microversion: 2.20 time: 0.352165 Oct 13 04:24:06.011878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 258/466] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:05 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1810 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:06.033604 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:06.034395 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:06.034992 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:06.036363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:06.036819 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:06.037405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:06.194842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:06.207050 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4d974c46-8c2e-42bf-8e8a-cfa44b67196b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1445 microversion: 2.1 time: 0.202742 Oct 13 04:24:06.215397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 209/467] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:06 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1445 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:06.549020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:06.549751 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:06.556916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:06.558048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:06.559056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:06.560454 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:06.561425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:06.563188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:06.832401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:06.848802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-67425cf0-32b7-4b31-bd3f-e1229dbcc1da tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1723 microversion: 2.1 time: 0.305309 Oct 13 04:24:06.849879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 259/468] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:06 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1723 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:07.046818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:07.047967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:07.054485 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:07.055298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:07.055903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:07.056611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:07.066087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:07.066772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:07.104888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:07.784770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:07.796355 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:07.798962 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:07.800401 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-564da45c-d7bc-4e5d-b2f3-c1a7776dbec5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1752 microversion: 2.20 time: 0.764977 Oct 13 04:24:07.801639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 210/469] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:07 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1752 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:07.875795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:07.876968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:07.888050 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:07.892220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:07.892677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:07.893899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:07.895836 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:07.896944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:08.420008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:08.436436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8f6a7f68-56c7-42e3-b6cb-d8f22b18f77b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1700 microversion: 2.1 time: 0.565880 Oct 13 04:24:08.438509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 260/470] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:07 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1700 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:08.469189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-357e9fa2-3b85-4606-9b6f-77121813b882 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:08.476970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-357e9fa2-3b85-4606-9b6f-77121813b882 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:08.477632 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-357e9fa2-3b85-4606-9b6f-77121813b882 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:08.478256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-357e9fa2-3b85-4606-9b6f-77121813b882 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:08.562423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-357e9fa2-3b85-4606-9b6f-77121813b882 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Going to try to start instance {{(pid=118674) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 13 04:24:08.635183 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-357e9fa2-3b85-4606-9b6f-77121813b882 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action" status: 202 len: 0 microversion: 2.1 time: 0.168987 Oct 13 04:24:08.637777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 211/471] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:24:08 2022] POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action => generated 0 bytes in 174 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:24:08.647605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:08.649059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:08.656737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:08.658462 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:08.659078 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:08.660114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:08.660558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:08.661092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:08.832958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:08.833665 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:08.848516 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:08.849020 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:08.849453 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:08.852605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:08.853091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:08.853527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:09.137119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:09.154042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a02335e5-ff15-4fe8-8f7f-17bb94f4b74a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1709 microversion: 2.1 time: 0.508921 Oct 13 04:24:09.157887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 261/472] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:08 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1709 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:09.442709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:09.458500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:09.462468 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:09.463880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-732e181a-0cf2-4283-880c-3e4660e44374 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.640000 Oct 13 04:24:09.466855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 212/473] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:08 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:10.187197 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:10.188192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:10.195246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:10.195968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:10.196594 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:10.197509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:10.198105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:10.198692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:10.496792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:10.497476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:10.524536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:10.524959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:10.525443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:10.534358 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:10.534945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:10.537418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:10.553565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:10.558129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:10.575939 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d8c1eeca-956c-4fa0-a2e9-1e7bf150f92f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1709 microversion: 2.1 time: 0.393718 Oct 13 04:24:10.576527 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 262/474] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:10 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1709 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:10.911056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-cef402a2-c00a-4168-b645-aa5861a48bae req-73436994-63cd-4e15-955b-bbea76978afa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:10.931966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-cef402a2-c00a-4168-b645-aa5861a48bae req-73436994-63cd-4e15-955b-bbea76978afa service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:10.932634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-cef402a2-c00a-4168-b645-aa5861a48bae req-73436994-63cd-4e15-955b-bbea76978afa service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:10.933134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-cef402a2-c00a-4168-b645-aa5861a48bae req-73436994-63cd-4e15-955b-bbea76978afa service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:10.960787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-cef402a2-c00a-4168-b645-aa5861a48bae req-73436994-63cd-4e15-955b-bbea76978afa service nova] Creating event network-changed:9b98fb07-be4c-4add-b232-a8149b2588d6 for instance f186151e-3b9a-4cdf-8d6e-427fea09de90 on hv-cinder-86099 Oct 13 04:24:10.968947 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-cef402a2-c00a-4168-b645-aa5861a48bae req-73436994-63cd-4e15-955b-bbea76978afa service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063489 Oct 13 04:24:10.970254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 263/475] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:11.020969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:11.033985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:11.035589 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:11.037145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2bd89590-dfe0-4220-ae79-af11259f82fc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.546671 Oct 13 04:24:11.038042 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 213/476] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:10 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:11.609391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:11.610522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:11.622565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:11.625854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:11.626534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:11.628663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:11.630914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:11.632619 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:11.914598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:11.923857 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cd2bdd43-ee93-46ac-a50d-378bf6ddf094 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1709 microversion: 2.1 time: 0.320421 Oct 13 04:24:11.924660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 264/477] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:11 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1709 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:12.063861 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:12.065080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:12.072888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:12.079771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:12.080434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:12.081329 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:12.082146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:12.082748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:12.451216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:12.465553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:12.475089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:12.482839 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-276b7ff7-7c92-4cf4-8624-b7791fb7433f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.419492 Oct 13 04:24:12.483547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 214/478] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:12 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:12.948812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:12.950175 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:12.963469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:12.966922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:12.971020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:12.972326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:12.972943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:12.973518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:13.280988 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:13.293397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-bf57edec-d61d-4ef7-bf0a-0528ab435c46 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1709 microversion: 2.1 time: 0.348785 Oct 13 04:24:13.294097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 265/479] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:12 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1709 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:13.516301 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:13.519051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:13.538730 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:13.546260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:13.547115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:13.548102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:13.548792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:13.549447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:13.840065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:13.849225 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:13.850694 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:13.851869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7ffa56ed-aa90-49ac-9175-b5c5bdeacd74 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.341933 Oct 13 04:24:13.862387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 215/480] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:13 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:14.322964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:14.323794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:14.336487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:14.337088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:14.337599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:14.338864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:14.342001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:14.342561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:14.638404 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:14.648002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2d125b17-0c74-4784-8899-3d2e48752073 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1713 microversion: 2.1 time: 0.330253 Oct 13 04:24:14.649511 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 266/481] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:14 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1713 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:14.883586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:14.884214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:14.892571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:14.893072 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:14.893505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:14.902036 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:14.905361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:14.905892 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:15.401464 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:15.414616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:15.417214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:15.419304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d796aa8f-2afe-4dc1-b96b-6f336bdeeab5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.540059 Oct 13 04:24:15.420271 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 216/482] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:14 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:16.445602 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:16.447478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:16.457354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:16.465789 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:16.466546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:16.467463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:16.468513 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:16.469312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:16.807848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:16.817871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:16.819441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:16.820633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3bd587f4-506f-473f-a55e-16ef2e3d5466 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.378990 Oct 13 04:24:16.821308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 267/483] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:16 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:17.852243 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:17.853815 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:17.865345 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:17.866777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:17.867553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:17.868421 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:17.869081 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:17.869650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:18.172008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:18.182524 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:18.184367 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:18.185909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ee20d55d-14aa-4645-b821-da7851aa830d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.339524 Oct 13 04:24:18.187305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 217/484] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:17 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:19.215624 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:19.216232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:19.225558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:19.226334 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:19.226937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:19.227804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:19.228263 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:19.228761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:19.550954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:19.561081 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:19.563538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:19.565253 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a2629227-916c-4190-b8e8-02d94e9fb901 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.353489 Oct 13 04:24:19.566422 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 268/485] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:19 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:20.595361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:20.596756 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:20.603894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:20.604552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:20.605246 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:20.606063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:20.606982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:20.607409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:20.768863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:20.779875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:20.783097 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:20.784836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5d067194-9076-4b66-a68f-98837e47557b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.194915 Oct 13 04:24:20.799115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 218/486] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:20 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:21.819905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:21.821608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:21.837285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:21.839224 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:21.842369 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:21.845707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:21.846645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:21.847477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:22.143299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:22.154194 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:22.156292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:22.157491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d0a8a5d4-b97a-45ba-b38e-6996f114cd1c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.347291 Oct 13 04:24:22.163637 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 269/487] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:21 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:23.190855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:23.191460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:23.207356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:23.209625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:23.210188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:23.211021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:23.211561 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:23.212118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:23.356555 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:23.366803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:23.368558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:23.369992 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3e04e6e8-6a0a-4250-bb3b-22c75a159c79 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.185250 Oct 13 04:24:23.382635 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 219/488] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:23 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:24.400622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:24.401997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:24.409872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:24.410737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:24.411400 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:24.412363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:24.413047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:24.413842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:24.576360 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:24.589494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:24.591065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:24.592329 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fc60d029-f82a-4a91-9625-3ad9c3294f2a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.197810 Oct 13 04:24:24.593546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 270/489] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:24 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:25.620956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:25.621523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:25.629109 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:25.629599 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:25.630035 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:25.630782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:25.631230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:25.631696 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:25.779516 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:25.789881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:25.791442 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:25.792982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-59cee5e2-f3ac-4e5a-bf00-9586befaf3cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.176157 Oct 13 04:24:25.793716 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 220/490] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:25 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:26.819920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:26.820517 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:26.826922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:26.827639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:26.828242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:26.829107 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:26.829712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:26.830324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:26.976671 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:26.986657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:26.988419 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:26.989665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a6d3cdf2-1d0a-462a-ac52-8b413ffaa0b3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.173699 Oct 13 04:24:26.990882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 271/491] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:26 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:28.016709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:28.018025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:28.025116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:28.025721 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:28.029050 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:28.029934 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:28.030698 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:28.031245 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:28.262501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:28.272887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:28.274336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:28.275503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f96d41a5-ca76-4c25-abd2-616a59226c9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.263674 Oct 13 04:24:28.276058 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 221/492] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:28 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:28.812351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9adae2e2-4e8a-4081-82e7-c2abb2dac508 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Action: 'create', calling method: .version_select at 0x7f07949fadc0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1798269318", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJbifU8LGwiZH/bzv+sQXBCRvklrP/lDaG02B4RgIBVq1cHBndKxAVN0mf26lGvopNLis9WyfjRMH7t11diySHm+8k0OPKcXHbo/013sUm/u7dX/ibvPRCuLrmtT5pWmPg=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:28.828957 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-9adae2e2-4e8a-4081-82e7-c2abb2dac508 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Getting quotas for project f2b13e92741f40bc8044ed4bc999b07f. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:24:28.832871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-9adae2e2-4e8a-4081-82e7-c2abb2dac508 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Getting quotas for user 7582f5c0de6b4b5dae18b0320921e4a2 and project f2b13e92741f40bc8044ed4bc999b07f. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:24:28.856545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9adae2e2-4e8a-4081-82e7-c2abb2dac508 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.048236 Oct 13 04:24:28.857150 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 272/493] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:24:28 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:28.908575 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1373872004", "imageRef": "8ae629a5-fd50-453c-ac23-15ca66c6cbf8", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1798269318", "networks": [{"uuid": "1ab6a10d-cef9-4781-9526-144afd708fcb"}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:28.959960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] validate_networks() for [('1ab6a10d-cef9-4781-9526-144afd708fcb', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:24:29.301742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:29.302379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:29.308137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:29.308610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:29.309045 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:29.309722 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:29.310168 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:29.310610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:29.330389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:24:29.330932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:24:29.331271 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:24:29.331678 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:24:29.332086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:24:29.340744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:29.341222 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:29.341654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:29.342418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:29.342871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:29.343327 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:29.360949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Getting quotas for project f2b13e92741f40bc8044ed4bc999b07f. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:24:29.364959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Getting quotas for user 7582f5c0de6b4b5dae18b0320921e4a2 and project f2b13e92741f40bc8044ed4bc999b07f. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:24:29.377030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:24:29.379314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='8ae629a5-fd50-453c-ac23-15ca66c6cbf8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:24:29.429538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-01fa801c-03a3-46e3-b3bb-8a6c29625100 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.525465 Oct 13 04:24:29.430403 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 222/494] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:24:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 527 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:24:29.441826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2faedd2f-5e42-49cf-95ac-94ece01394ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:29.442469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2faedd2f-5e42-49cf-95ac-94ece01394ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:29.445473 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:29.455903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:29.468241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:29.469552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-70e26bfa-4e04-4d5d-a327-a3257538ce6f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.171855 Oct 13 04:24:29.470197 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 273/495] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:29 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:29.496797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2faedd2f-5e42-49cf-95ac-94ece01394ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:29.503662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2faedd2f-5e42-49cf-95ac-94ece01394ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1259 microversion: 2.1 time: 0.065705 Oct 13 04:24:29.504236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 223/496] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:29 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1259 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:30.496513 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:30.497120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:30.503675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:30.504100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:30.504538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:30.505255 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:30.505706 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:30.506127 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:30.525655 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:30.526302 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:30.533585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:30.534073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:30.534542 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:30.535236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:30.535688 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:30.536116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:30.600696 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:30.609193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4a7d9b0a-c7be-4c49-969f-8e764b5d8451 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1263 microversion: 2.1 time: 0.087279 Oct 13 04:24:30.609763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 224/497] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:30 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1263 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:30.641351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:30.649917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:30.651407 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:30.652691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e3b1584c-12fb-4881-95f7-c5ca33f8f582 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.160087 Oct 13 04:24:30.653254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 274/498] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:30 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:31.633444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:31.634398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:31.641748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:31.642261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:31.642703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:31.643404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:31.643858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:31.644283 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:31.679581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:31.680218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:31.682726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:31.689500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:31.689990 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:31.690449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:31.691152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:31.691600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:31.692017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:31.711484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:31.722491 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f917a793-ff11-4179-a57c-647bb6489129 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1311 microversion: 2.1 time: 0.094505 Oct 13 04:24:31.723663 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 225/499] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:31 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1311 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:31.847602 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:31.857299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:31.859144 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:31.860359 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-599c6d60-5339-42c2-a0f7-5b9d42af0b24 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.184916 Oct 13 04:24:31.861103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 275/500] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:31 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:32.747434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:32.749352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:32.753530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:32.760248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:32.760873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:32.761414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:32.762288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:32.762984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:32.763623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:32.897591 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:32.899561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:32.906729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:32.907569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:32.908235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:32.909161 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:32.909896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:32.910588 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:32.988113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:32.995989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ced71c0-daf8-41ca-a6fc-a0ef93bb10a8 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.252675 Oct 13 04:24:32.996653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 226/501] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:32 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:33.063687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:33.073106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:33.074628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:33.075816 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3be0519c-b7b4-4b7a-af39-91ee1aeff29d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.188975 Oct 13 04:24:33.076919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 276/502] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:32 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:33.832884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-6c66b821-3d88-4f56-9cab-477d7eee94a4 req-b15d17dc-811b-4273-81a5-0e1343e2a888 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "name": "network-vif-plugged", "status": "completed", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:33.840507 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 13 04:24:33.840825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Thu Oct 13 04:24:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.27) !!! Oct 13 04:24:33.849362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-6c66b821-3d88-4f56-9cab-477d7eee94a4 req-b15d17dc-811b-4273-81a5-0e1343e2a888 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:33.849842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-6c66b821-3d88-4f56-9cab-477d7eee94a4 req-b15d17dc-811b-4273-81a5-0e1343e2a888 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:33.850293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-6c66b821-3d88-4f56-9cab-477d7eee94a4 req-b15d17dc-811b-4273-81a5-0e1343e2a888 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:33.863547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-6c66b821-3d88-4f56-9cab-477d7eee94a4 req-b15d17dc-811b-4273-81a5-0e1343e2a888 service nova] Creating event network-vif-plugged:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:24:33.868791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-6c66b821-3d88-4f56-9cab-477d7eee94a4 req-b15d17dc-811b-4273-81a5-0e1343e2a888 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040325 Oct 13 04:24:33.869707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 227/503] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:24:34.019833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:34.020575 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:34.027177 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:34.027668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:34.028105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:34.028807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:34.029521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:34.029950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:34.102072 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:34.103130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:34.110251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:34.110752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:34.111214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:34.111953 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:34.112406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:34.112830 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:34.117514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:34.176700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:34.187738 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5f3d339c-c09f-4553-8b23-721b458dc854 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.171685 Oct 13 04:24:34.189004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 277/504] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:34 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:34.262753 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:34.271960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:34.273409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:34.274616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cfb0643e-d1ce-4cbe-b997-e772aaed45b2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.177051 Oct 13 04:24:34.275370 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 228/505] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:34 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:35.211691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:35.212296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:35.220162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:35.220640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:35.221069 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:35.221762 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:35.222224 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:35.222652 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:35.300797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:35.301400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:35.308247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:35.308734 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:35.309354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:35.310061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:35.310548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:35.311185 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:35.360626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:35.375411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c7ff6216-6037-4376-8e49-295bfd0ad6fe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.167488 Oct 13 04:24:35.375995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 278/506] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:35 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:35.463116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:35.472366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:35.473860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:35.475028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d9e0bf18-929b-4b33-8a23-e5f938357ed0 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.178130 Oct 13 04:24:35.475586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 229/507] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:35 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:36.398673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:36.399359 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:36.405846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:36.407157 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:36.407835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:36.408709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:36.409183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:36.409632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:36.503141 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:36.505054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:36.512554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:36.513389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:36.514064 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:36.515285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:36.516634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:36.517279 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:36.566015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:36.582920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cd92358a-7296-4c39-bbac-0675e74cfbae tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.187987 Oct 13 04:24:36.583780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 279/508] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:36 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:36.678739 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:36.687501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:36.689721 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:36.690933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-97376eec-2269-4e27-9d59-7c2be496f688 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.193227 Oct 13 04:24:36.691685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 230/509] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:36 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:37.607599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:37.608434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:37.615282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:37.615877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:37.621824 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.622363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.622854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.629317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.630260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.630988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.634545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.635425 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.636189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.646836 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.647499 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.647967 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.674744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a2efec72-e206-43a8-bf93-acb5a7f42a70 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/ips" status: 200 len: 150 microversion: 2.1 time: 0.070306 Oct 13 04:24:37.675975 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 231/510] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:24:37 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/ips => generated 150 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:37.687430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ccd0cb13-8965-4bc8-a8c4-3c35c2995160"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:37.690433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:37.698265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.698265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.698265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.698265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.698735 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.699136 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.803106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Invalid volume: volume ccd0cb13-8965-4bc8-a8c4-3c35c2995160 is already attached to instances: 62033c3e-e894-4560-95b8-34e25cb33e72 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: POST /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/os-volume_attachments HTTP/1.1 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 74 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:24:37.804841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {"volumeAttachment": {"volumeId": "ccd0cb13-8965-4bc8-a8c4-3c35c2995160"}} Oct 13 04:24:37.806466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Invalid volume: volume ccd0cb13-8965-4bc8-a8c4-3c35c2995160 is already attached to instances: 62033c3e-e894-4560-95b8-34e25cb33e72 Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'server_id': 'f3720a7c-f5a9-468e-8195-221790e8931c', 'body': {'volumeAttachment': {'volumeId': 'ccd0cb13-8965-4bc8-a8c4-3c35c2995160'}}} Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume ccd0cb13-8965-4bc8-a8c4-3c35c2995160 is already attached to instances: 62033c3e-e894-4560-95b8-34e25cb33e72 Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:24:37.807554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume ccd0cb13-8965-4bc8-a8c4-3c35c2995160 is already attached to instances: 62033c3e-e894-4560-95b8-34e25cb33e72 Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:24:37.809496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume ccd0cb13-8965-4bc8-a8c4-3c35c2995160 is already attached to instances: 62033c3e-e894-4560-95b8-34e25cb33e72 Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 400 to user: Invalid volume: volume ccd0cb13-8965-4bc8-a8c4-3c35c2995160 is already attached to instances: 62033c3e-e894-4560-95b8-34e25cb33e72 {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-da6ee7ba-43cf-46b4-b0dc-723f3fbf3073 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.122949 Oct 13 04:24:37.811999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 232/511] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:24:37 2022] POST /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c/os-volume_attachments => generated 174 bytes in 124 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 13 04:24:37.813064 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:37.813064 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:37.820541 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:37.822297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.822802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.823237 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.823945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.824392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.824813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.829202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-77d7fb15-2ea5-44da-afa5-a06c90dc3535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.227911 Oct 13 04:24:37.830435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 280/512] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:37 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:37.840333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:37.842854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:37.849894 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.850760 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.851312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.852145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:37.852853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:37.853379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:37.858667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:37.885839 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:24:37.967842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:37.979871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:37.981830 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:37.983361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4861246b-6093-4878-b088-206e43e04ecc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.175206 Oct 13 04:24:37.983972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 233/513] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:37 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:37.991120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-adae7aba-9d2b-4fa4-a926-a63cd3cb1c42 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 204 len: 0 microversion: 2.1 time: 0.159018 Oct 13 04:24:37.991973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 281/514] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:24:37 2022] DELETE /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:24:37.999975 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:38.000768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:38.007905 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:38.008390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:38.008822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:38.009639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:38.010121 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:38.010611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:38.153586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:38.162711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ed1ff69a-87ca-49c2-9798-166697c5e090 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.166308 Oct 13 04:24:38.163290 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 234/515] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:37 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:38.853757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:38.855337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:38.858304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:38.864941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:38.865732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:38.866261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:38.867036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:38.867607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:38.868107 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:39.007431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:39.013267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:39.014513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:39.015581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f007187a-599b-4e5a-8f31-bdd09d4b3f50 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.166134 Oct 13 04:24:39.021924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:39.022692 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:39.023280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:39.023980 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:39.024429 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:39.024847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:39.030712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 282/516] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:38 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:39.161669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:39.170956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:39.172391 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:39.173639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8fda21ef-8191-4642-8db8-cfc28931fc33 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.166614 Oct 13 04:24:39.174416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 235/517] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:39 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:39.192838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:39.193640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:39.200002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:39.200492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:39.200931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:39.201672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:39.202098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:39.202549 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:39.336907 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:39.348474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6086443f-c977-49a0-ad1b-445207234cbc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.161159 Oct 13 04:24:39.349060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 283/518] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:39 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:40.039558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:40.040186 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:40.041894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:40.042886 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=8 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:40.049354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:40.049845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:40.050310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:40.051017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:40.051470 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:40.051898 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:40.189379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:40.197707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4bfed7d9-360e-48be-80ea-3c4123ff17a3 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.162074 Oct 13 04:24:40.198306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 236/519] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:40 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:40.202952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:40.203555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:40.211610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:40.212316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:40.212907 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:40.213795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:40.214434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:40.214853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:40.352723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:40.362917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:40.364562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:40.365820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5b820117-543a-4fbc-9d20-e95f25e77975 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.167263 Oct 13 04:24:40.370256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 284/520] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:40 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:40.373700 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:40.374536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:40.384085 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:40.384566 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:40.384994 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:40.385691 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:40.386136 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:40.386588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:40.526547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:40.534828 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-dd24f13b-0db5-4a54-9760-57cce0c0bb57 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.165017 Oct 13 04:24:40.535600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 237/521] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:40 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:41.221641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:41.222519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:41.230444 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.230677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4b9f29a3-4a06-4419-ae33-c3c1908cd6ac"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:41.232075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.232903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.233466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:41.234122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.234832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.235427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.241157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.241883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.242348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.243157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.243605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.244285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.401854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:41.420892 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cee620ae-9d7d-408f-9ebe-1c087019389d tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.203767 Oct 13 04:24:41.421803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 285/522] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:41 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:41.427075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:41.427685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:41.433766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.434294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.434734 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.435432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.435883 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.436300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.571059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:41.580351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:41.581826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:41.583023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-47f83f87-dd2c-4047-9eed-37e3877f42c6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.160255 Oct 13 04:24:41.583574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 286/523] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:41 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:41.587898 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:41.588494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:41.594226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.594722 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.595209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.595897 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:41.596349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:41.596770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:41.750174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:41.758681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9b87f3c6-b52e-4a75-ae2b-b63120823e93 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.174075 Oct 13 04:24:41.759267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 287/524] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:41 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:42.444479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:42.445068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:42.451569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:42.452051 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:42.452484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:42.453170 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:42.453620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:42.454077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:42.581158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:42.588806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-852c28d4-e789-425a-9905-8c098b7afb10 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.148217 Oct 13 04:24:42.589379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 288/525] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:42 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:42.611551 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:42.612146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:42.617689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:42.618173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:42.618640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:42.619335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:42.619790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:42.620216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:42.749401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:42.758526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:42.759965 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:42.761126 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ea819a8f-6679-4799-b133-0f231ed611e6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.153177 Oct 13 04:24:42.761684 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 289/526] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:42 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:42.783027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:42.783565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:42.789199 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:42.789676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:42.790110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:42.790826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:42.791274 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:42.791694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:42.926960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:42.934817 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-172891e1-ae3d-45d1-b6ba-46667228e40b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.155619 Oct 13 04:24:42.941865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 290/527] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:42 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:43.526054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-d3075f1e-b255-41e5-89d2-be0726fa7a86 req-d6570d29-2157-4762-9a6d-4f61cfee0c98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:43.536194 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d3075f1e-b255-41e5-89d2-be0726fa7a86 req-d6570d29-2157-4762-9a6d-4f61cfee0c98 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.536675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d3075f1e-b255-41e5-89d2-be0726fa7a86 req-d6570d29-2157-4762-9a6d-4f61cfee0c98 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.537110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d3075f1e-b255-41e5-89d2-be0726fa7a86 req-d6570d29-2157-4762-9a6d-4f61cfee0c98 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:43.548541 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-d3075f1e-b255-41e5-89d2-be0726fa7a86 req-d6570d29-2157-4762-9a6d-4f61cfee0c98 service nova] Creating event network-changed:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:24:43.553395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-d3075f1e-b255-41e5-89d2-be0726fa7a86 req-d6570d29-2157-4762-9a6d-4f61cfee0c98 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030989 Oct 13 04:24:43.554190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 291/528] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:24:43.613639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:43.614381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:43.620393 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.620871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.621300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:43.621984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.622456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.622874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:43.751399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:43.759181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b3da6ac4-75cc-42a3-9807-8509fad16bd1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.150387 Oct 13 04:24:43.759746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 292/529] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:43 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:43.786273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:43.786869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:43.792428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.792901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.793332 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:43.794006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.794617 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.795040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:43.925739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:43.934335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:43.935771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:43.936911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b4f7bb6d-ba74-4fa3-bd3d-600b5356e483 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.155035 Oct 13 04:24:43.937457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 293/530] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:43 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:43.957698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:43.958253 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:43.963891 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.964366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.964792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:43.965478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:43.965924 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:43.966374 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:44.093603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:44.101905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d76fcf4f-2f44-422d-bad8-603a2800e9a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.147871 Oct 13 04:24:44.102546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 294/531] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:43 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:44.780860 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:44.781488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:44.787959 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:44.788439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:44.788871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:44.789556 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:44.790004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:44.790456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:44.920908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:44.929244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1a4408e8-1446-4d6d-9d09-faad06114bc9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.152229 Oct 13 04:24:44.929822 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 295/532] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:44 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:44.962781 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:44.963835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:44.970400 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:44.970896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:44.971334 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:44.972229 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:44.972693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:44.973282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:45.072447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-79ac67bc-9681-41d6-a25d-d6d000f93d81 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.846536 Oct 13 04:24:45.073372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 238/533] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:24:41 2022] POST /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments => generated 194 bytes in 3848 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:45.125192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:45.127244 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:45.128151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:45.131585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:45.135278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:45.135952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:45.136540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:45.137392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:45.138019 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:45.138686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:45.139128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:45.140971 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:45.142484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7a76e35c-792f-49c6-a14f-716391e5ff50 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.183542 Oct 13 04:24:45.143496 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 296/534] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:44 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:45.576078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:45.584168 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d18f97bc-a572-4a45-8004-49ff7e52a91a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1681 microversion: 2.1 time: 0.462319 Oct 13 04:24:45.584746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 239/535] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:45 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1681 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:45.952592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:45.953199 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:45.960268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:45.960750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:45.961185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:45.961871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:45.962350 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:45.962783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:46.099468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:46.107876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1fab2abb-0362-4dc2-bff0-5fe60e757242 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.159041 Oct 13 04:24:46.108484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 297/536] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:45 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:46.170357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:46.171427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:46.179097 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:46.179587 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:46.180106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:46.181249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:46.181492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:46.181785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:46.564916 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:46.574164 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:46.575948 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:46.577165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-af2a2024-8909-422d-94b3-706b0e0a91f5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.410862 Oct 13 04:24:46.577762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 240/537] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:46 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:46.609242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:46.610083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:46.616340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:46.616916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:46.617421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:46.618185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:46.618748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:46.619240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:46.685027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:46.694198 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fc2137e1-b10b-451d-abec-f51a19f1c9d1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1613 microversion: 2.1 time: 0.088430 Oct 13 04:24:46.694498 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 298/538] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:46 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1613 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:46.765183 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-1a8f25f2-e77a-49e0-8d10-9112328c61cb req-ced41109-b39c-479e-b924-d96cb9e5ad60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3720a7c-f5a9-468e-8195-221790e8931c", "name": "network-vif-deleted", "tag": "fb3e21a4-2e2b-468a-8003-e487721c3939"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:46.776310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1a8f25f2-e77a-49e0-8d10-9112328c61cb req-ced41109-b39c-479e-b924-d96cb9e5ad60 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:46.776900 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1a8f25f2-e77a-49e0-8d10-9112328c61cb req-ced41109-b39c-479e-b924-d96cb9e5ad60 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:46.777396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1a8f25f2-e77a-49e0-8d10-9112328c61cb req-ced41109-b39c-479e-b924-d96cb9e5ad60 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:46.789887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-1a8f25f2-e77a-49e0-8d10-9112328c61cb req-ced41109-b39c-479e-b924-d96cb9e5ad60 service nova] Creating event network-vif-deleted:fb3e21a4-2e2b-468a-8003-e487721c3939 for instance f3720a7c-f5a9-468e-8195-221790e8931c on hv-cinder-86099 Oct 13 04:24:46.794920 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-1a8f25f2-e77a-49e0-8d10-9112328c61cb req-ced41109-b39c-479e-b924-d96cb9e5ad60 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034225 Oct 13 04:24:46.795860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 241/539] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:24:47.132408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:47.133600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:47.139946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.140632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.141242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.141997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.142500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.142930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.288917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:47.299769 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9984a5a1-89df-4b3f-baf6-a11a51a94abe tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.171170 Oct 13 04:24:47.300435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 299/540] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:47 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:47.630750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:47.649567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:47.656280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.657265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.657889 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.658923 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.659528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.660356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.719667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:47.720514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:47.728789 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.729351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.730464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.731574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.732350 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.733085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.788362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:47.789095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:47.789534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:47.790476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid f3720a7c-f5a9-468e-8195-221790e8931c {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:24:47.848381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:47.862820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1008f724-74a9-4b14-8fd6-e9f4c0606b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1330 microversion: 2.1 time: 0.147972 Oct 13 04:24:47.863280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 300/541] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:47 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1330 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:47.889330 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:47.900779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:47.902624 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:47.904174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7ad3b955-593f-4fcc-8352-1a103ef36cb1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.302284 Oct 13 04:24:47.905949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 242/542] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:47 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:48.322784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:48.323394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:48.330401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.330888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.331322 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.332040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.332493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.332915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.471851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:48.479418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-48afcb5e-4fc7-4aa8-a5cf-899f55489e3a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.160607 Oct 13 04:24:48.479979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 301/543] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:48 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:48.888367 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:48.888884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:48.895781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.896249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.896712 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.897347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.897793 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.898225 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.928463 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.928942 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.929374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.929812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid f3720a7c-f5a9-468e-8195-221790e8931c {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:24:48.930791 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:48.931399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:48.937312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.937804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.938846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.939564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:48.940013 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:48.940436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:48.972005 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:48.980246 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-56a54118-2836-41d8-b953-3df2cdad84ae tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 200 len: 1330 microversion: 2.1 time: 0.095712 Oct 13 04:24:48.980825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 243/544] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:48 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 1330 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:49.089354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:49.106682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:49.108940 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:49.110339 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-63060ecb-3392-443e-9fa4-5808b4750e70 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1808 microversion: 2.20 time: 0.183402 Oct 13 04:24:49.111505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 302/545] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:48 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1808 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:24:49.502979 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:49.503880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:49.510004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:49.510517 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:49.510951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:49.511673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:49.512127 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:49.512549 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:49.517338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:49.648033 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:49.655594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c7a71dfd-fd48-430f-bd96-e023f535ed0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.156901 Oct 13 04:24:49.656159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 244/546] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:49 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:50.003937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:50.004546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:50.011131 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.011612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.012047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.012738 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.013193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.013615 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.028720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c HTTP/1.1 Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:24:50.030375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:24:50.031904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'f3720a7c-f5a9-468e-8195-221790e8931c'} Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:24:50.033456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:24:50.034979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:24:50.036347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:24:50.036347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f408e225-a8e8-41f3-a442-cde60b816da5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 404 len: 111 microversion: 2.1 time: 0.031888 Oct 13 04:24:50.036347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 303/547] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:49 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:24:50.041213 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d2d5bef8-5e77-463e-ba43-b3f033a411f0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '.version_select at 0x7f0795ebff70>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:50.063046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d2d5bef8-5e77-463e-ba43-b3f033a411f0 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-715085596" status: 202 len: 0 microversion: 2.1 time: 0.025623 Oct 13 04:24:50.063603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 245/548] 10.222.0.27 () {58 vars in 1218 bytes} [Thu Oct 13 04:24:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-715085596 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:24:50.137307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:50.137853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:50.144461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.144944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.145379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.146073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.146547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.146980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.291200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:50.301004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:50.302518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:50.303730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0eeca275-2a6e-4f2d-9bab-521817613903 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1795 microversion: 2.20 time: 0.170446 Oct 13 04:24:50.304290 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 304/549] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:24:50 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1795 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:24:50.679741 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:50.680340 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:50.687185 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.687666 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.688106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.688995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.689351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.689768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.765705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:50.766215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:50.772870 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.773346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.773968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.774772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:50.775329 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:50.775775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:50.825859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:50.833568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-90b44c91-08c3-4e39-b71d-66c985cb308a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.157801 Oct 13 04:24:50.834136 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 246/550] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:50 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:50.901730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'services' on Instance uuid 62033c3e-e894-4560-95b8-34e25cb33e72 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:24:51.013308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ff05b73a-e2a6-451f-9eaf-e264339991b2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments/ccd0cb13-8965-4bc8-a8c4-3c35c2995160" status: 202 len: 0 microversion: 2.1 time: 0.251455 Oct 13 04:24:51.014040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 305/551] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:24:50 2022] DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments/ccd0cb13-8965-4bc8-a8c4-3c35c2995160 => generated 0 bytes in 253 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:24:51.022424 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:51.024540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:51.030813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:51.031299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:51.031732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:51.032688 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:51.033182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:51.033598 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:51.063959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e16f55bb-3e34-4cca-87b0-8913ec003826 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044778 Oct 13 04:24:51.064777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 247/552] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:24:51 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:24:51.857206 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:51.857902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:51.865683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:51.866163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:51.866623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:51.867323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:51.867779 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:51.868200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.014469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:52.022941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-37e020d3-8858-4351-9fed-7a7fae0a14e5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.170410 Oct 13 04:24:52.023561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 306/553] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:51 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:52.076985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:52.080067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:52.086473 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:52.086956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:52.087437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.088117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:52.088562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:52.088978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.119170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f426bfba-a2cd-4ccd-8735-76a5e0e3eff2 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046192 Oct 13 04:24:52.119721 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 248/554] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:24:52 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:52.660328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:52.662704 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:52.668721 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:52.669155 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:52.669590 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.670309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:52.670766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:52.671208 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.697001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'flavor' on Instance uuid 168caf0b-e6d8-441d-adfd-821ed4238c11 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:24:52.817588 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d3ccb53b-808c-4cbe-a7f6-9898b3c1d24e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/action" status: 200 len: 32 microversion: 2.1 time: 0.161830 Oct 13 04:24:52.818242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 307/555] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:24:52 2022] POST /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/action => generated 32 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:24:52.827570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:52.828178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:52.834442 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:52.834925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:52.835359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.836037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:52.836490 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:52.836909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:52.964371 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:52.972737 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4fcb0b18-ee93-4fd1-9fc2-27e0400bc7bc tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.148914 Oct 13 04:24:52.973298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 249/556] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:52 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:53.047478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:53.049174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:53.056455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:53.056935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:53.057365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:53.058052 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:53.058530 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:53.059051 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:53.133625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:53.136012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:53.144179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:53.144725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:53.145317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:53.146134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:53.146611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:53.147104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:53.179676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3f0962a7-898d-4cde-be10-1f75e76e629a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050552 Oct 13 04:24:53.180660 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 250/557] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:24:53 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:24:53.207388 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:53.215226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2466e62c-5de9-4f35-af23-ab381052d2f1 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.172108 Oct 13 04:24:53.215718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 308/558] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:53 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:53.929781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-3b40ade7-6368-4d24-8108-dacc63295119 req-aa005cce-2373-4858-885c-908be217885d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "name": "network-vif-plugged", "status": "completed", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:53.940812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-3b40ade7-6368-4d24-8108-dacc63295119 req-aa005cce-2373-4858-885c-908be217885d service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:53.941292 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-3b40ade7-6368-4d24-8108-dacc63295119 req-aa005cce-2373-4858-885c-908be217885d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:53.941875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-3b40ade7-6368-4d24-8108-dacc63295119 req-aa005cce-2373-4858-885c-908be217885d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:53.952820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-3b40ade7-6368-4d24-8108-dacc63295119 req-aa005cce-2373-4858-885c-908be217885d service nova] Creating event network-vif-plugged:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:24:53.958715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-3b40ade7-6368-4d24-8108-dacc63295119 req-aa005cce-2373-4858-885c-908be217885d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032572 Oct 13 04:24:53.959549 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 251/559] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:53.997363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:53.998042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:54.004549 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.005043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.005508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.006272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.006731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.007192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.142793 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:54.150938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ca2206a5-b56a-4fa6-bc6c-095023b2fbb1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.157967 Oct 13 04:24:54.151502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 309/560] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:53 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:54.193366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:54.195642 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:54.202141 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.202673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.203105 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.204384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.204875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.205299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.236249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-74970d1e-1f99-4ee0-bb4a-803b1a20d11e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047903 Oct 13 04:24:54.236996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 252/561] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:24:54 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:24:54.237111 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:54.237796 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:54.244184 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.244499 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.245030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.245808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.246369 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.246866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.302715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:54.303209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:54.309060 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.309547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.309986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.310717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.311156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.311584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.337406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:24:54.377946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:54.385605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9431c5f3-4fe1-4bd6-b1f6-35d7c778c4ff tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.152443 Oct 13 04:24:54.386389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 310/562] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:54 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:54.420836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b00c80d6-9b06-451e-996e-0a68b890bfee tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 204 len: 0 microversion: 2.1 time: 0.121647 Oct 13 04:24:54.421422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 253/563] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:24:54 2022] DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 13 04:24:54.429960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:54.430828 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:54.436753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.437231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.437676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.438490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:54.438942 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:54.439362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:54.443308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:24:54.567772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:54.575439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b0df914d-a95e-426e-a1f1-22e517187d22 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1683 microversion: 2.1 time: 0.149245 Oct 13 04:24:54.576009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 311/564] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:54 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1683 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:55.175599 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:55.176323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:55.182539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:55.183023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:55.183454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:55.184144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:55.184596 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:55.185016 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:55.315750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:55.323818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2ed823b9-8495-4ae0-be5a-54ee431ffc0f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.153277 Oct 13 04:24:55.324376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 254/565] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:55 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:55.410606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:55.411214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:55.417681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:55.418453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:55.418701 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:55.419396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:55.419844 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:55.420260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:55.552173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:55.559742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ed9d7a9a-9cbe-475e-a148-a0dbc7d5ca0b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.153696 Oct 13 04:24:55.560357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 312/566] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:55 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:55.598736 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:55.599349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:55.605344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:55.605814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:55.606547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:55.607231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:55.607674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:55.608117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:55.736803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:55.744502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fbda98f7-0f6d-4621-90d4-72eadbe05da4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1683 microversion: 2.1 time: 0.150198 Oct 13 04:24:55.745247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 255/567] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:55 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1683 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:56.346930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:56.347538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:56.353558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:56.354041 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:56.354501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:56.355192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:56.355646 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:56.356072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:56.486383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:56.494563 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-74785795-2974-42bb-80b3-059d769398db tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.151470 Oct 13 04:24:56.495130 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 313/568] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:56 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:56.584126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:56.584732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:56.591855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:56.592344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:56.592777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:56.593471 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:56.593920 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:56.594908 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:56.736310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:56.744087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-676fa4d4-7cfb-46b0-8d44-8b9b292efca4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1359 microversion: 2.1 time: 0.164230 Oct 13 04:24:56.744664 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 256/569] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:56 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:56.769377 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:56.770881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:56.775976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:56.776456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:56.776887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:56.777576 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:56.778025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:56.778473 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:56.923605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:56.931763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-672b955c-91a1-48d1-a3a1-6acd35007ea1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1683 microversion: 2.1 time: 0.167255 Oct 13 04:24:56.932371 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 314/570] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:56 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1683 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:57.520809 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:57.521435 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:57.531344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.531927 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.532680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.533840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.534328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.534752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.691891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:57.700016 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-de700748-0b8a-49b0-acc7-d6eb95e01cde tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.183901 Oct 13 04:24:57.700582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 257/571] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:57 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:57.768572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:57.769174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:57.775568 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.776042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.776466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.777144 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.777590 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.778006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.905183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:57.913356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ed4d7d93-b60b-425a-8718-3a1adbecfe4f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1550 microversion: 2.1 time: 0.148649 Oct 13 04:24:57.913924 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 315/572] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:57 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1550 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:57.935930 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:57.936536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:57.943096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.943821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.944247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.945134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.945582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.946000 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.956408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:57.957015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:57.962923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.963407 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.963832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:57.964622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:57.965074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:57.965493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.097761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:58.105599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-edba5db0-b40a-4c14-8679-04f2331f40f1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1683 microversion: 2.1 time: 0.153279 Oct 13 04:24:58.106001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 316/573] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:57 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1683 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:24:58.174007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:58.181752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e709674b-3b55-485a-a64b-4c0ff4831346 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1550 microversion: 2.1 time: 0.249663 Oct 13 04:24:58.182435 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 258/574] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:57 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1550 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:58.204469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:58.209468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:58.210245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Searching by: {'deleted': False, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f'} {{(pid=118673) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 13 04:24:58.217209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.217689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.218121 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.218865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.219416 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.219992 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.319682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.multi_cell_list [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=118673) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 13 04:24:58.418798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2ac9cd03-8e77-4725-8a22-b0c8fd918e1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.218620 Oct 13 04:24:58.419784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 317/575] 10.222.0.27 () {58 vars in 1125 bytes} [Thu Oct 13 04:24:58 2022] GET /compute/v2.1/servers => generated 343 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:58.430879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:58.431416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:58.437197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.437681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.438314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.439161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.439620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.440045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.674654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:58.682445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-95c4837f-3c93-477a-80de-d3ba7489ff67 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1550 microversion: 2.1 time: 0.255282 Oct 13 04:24:58.682945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 259/576] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:58 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1550 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:58.728085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:58.729005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:58.736216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.736861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.737295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.737984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.738455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.739147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.887752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:58.901496 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-49dee7e4-b683-49cc-94dd-7595547961f4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.177629 Oct 13 04:24:58.903097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 318/577] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:58 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:58.932287 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-6bc0d87d-7e8d-447c-9ffb-57de939b127d req-851eb1e9-75dd-47cd-8af1-23104340ed92 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62033c3e-e894-4560-95b8-34e25cb33e72", "name": "network-vif-deleted", "tag": "bbeaed93-8e22-4d9f-a3ed-cb1f27426334"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:24:58.969896 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-6bc0d87d-7e8d-447c-9ffb-57de939b127d req-851eb1e9-75dd-47cd-8af1-23104340ed92 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:58.970448 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-6bc0d87d-7e8d-447c-9ffb-57de939b127d req-851eb1e9-75dd-47cd-8af1-23104340ed92 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:58.970906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-6bc0d87d-7e8d-447c-9ffb-57de939b127d req-851eb1e9-75dd-47cd-8af1-23104340ed92 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:58.984061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-6bc0d87d-7e8d-447c-9ffb-57de939b127d req-851eb1e9-75dd-47cd-8af1-23104340ed92 service nova] Creating event network-vif-deleted:bbeaed93-8e22-4d9f-a3ed-cb1f27426334 for instance 62033c3e-e894-4560-95b8-34e25cb33e72 on hv-cinder-86099 Oct 13 04:24:58.993270 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-6bc0d87d-7e8d-447c-9ffb-57de939b127d req-851eb1e9-75dd-47cd-8af1-23104340ed92 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065488 Oct 13 04:24:58.994561 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 260/578] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:24:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:24:59.132214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:59.133426 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:59.142996 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:59.143677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:59.145200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:59.146074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:59.146943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:59.147380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:59.191613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:59.192005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:59.192438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:59.192887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid 62033c3e-e894-4560-95b8-34e25cb33e72 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:24:59.237803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:24:59.247183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a27fa24c-8a81-40f7-b38a-18a54f910624 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 200 len: 1332 microversion: 2.1 time: 0.120244 Oct 13 04:24:59.247851 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 319/579] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:59 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 1332 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:24:59.931649 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:24:59.932357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:24:59.938994 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:59.939522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:59.940111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:59.940868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:24:59.941476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:24:59.941905 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:24:59.969255 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:00.088749 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:00.098115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c97dbd94-823d-4272-b510-6d6b138adbae tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.172009 Oct 13 04:25:00.098797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 261/580] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:24:59 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:00.272995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:00.273608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:00.275998 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:00.281361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:00.281854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:00.282322 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:00.283031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:00.283487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:00.283911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:00.297874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 HTTP/1.1 Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:25:00.299647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:25:00.300877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '62033c3e-e894-4560-95b8-34e25cb33e72'} Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:25:00.301569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:25:00.302716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:00.303750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:25:00.303750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-270cef54-f76d-4016-ab88-ceba6807e5f9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 404 len: 111 microversion: 2.1 time: 0.031828 Oct 13 04:25:00.303750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 320/581] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:00 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:25:00.312253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aac94479-7fd7-4044-888f-713ebf72fac5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '.version_select at 0x7f0794a31700>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:00.315758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f3c45e48-4d78-444c-95c8-596a79d73bd5"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:00.318072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:00.323900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:00.324386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:00.324820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:00.325512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:00.325963 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:00.326459 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:00.333536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aac94479-7fd7-4044-888f-713ebf72fac5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1115896741" status: 202 len: 0 microversion: 2.1 time: 0.025005 Oct 13 04:25:00.334292 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 262/582] 10.222.0.27 () {58 vars in 1221 bytes} [Thu Oct 13 04:25:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1115896741 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:25:00.625050 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1a265c53-b5cf-4710-8278-90da0118369c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.313494 Oct 13 04:25:00.626569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 321/583] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:25:00 2022] POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments => generated 194 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:25:00.984025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-41525761-dbea-4e9c-ad1b-cb2de15f1d96 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: .version_select at 0x7f07949d8700>, body: {"keypair": {"name": "tempest-keypair-383762571", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGwN2DJ77rim8jeOmaNC6fC8O2awsC77QQOEKIRKBGQanBuEodiiP4vX2Zz8BdkTYEB1DiHeKz71byoota2GF1Zg87XiLGJ+ju2O//gIpkTIPGl0MDOgQutBgHHIYKrIUQ=="}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:01.000834 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-41525761-dbea-4e9c-ad1b-cb2de15f1d96 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:25:01.004645 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-41525761-dbea-4e9c-ad1b-cb2de15f1d96 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:25:01.026815 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-41525761-dbea-4e9c-ad1b-cb2de15f1d96 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.046868 Oct 13 04:25:01.027415 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 263/584] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:25:00 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:01.122488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:01.123176 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:01.131622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:01.132431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:01.133010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:01.134448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:01.135488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:01.136343 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:01.280030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:01.289719 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5a0f3729-1dac-46bc-9290-ff3ad6a9cd76 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.171786 Oct 13 04:25:01.290483 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 322/585] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:01 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:02.298046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1913628262", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "70d105db-93f5-4592-bb80-f58af13e321b"}], "security_groups": [{"name": "tempest-securitygroup--2048190254"}], "key_name": "tempest-keypair-383762571", "user_data": "}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:02.314145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:02.314785 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:02.322416 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:02.322907 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:02.323341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:02.324032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:02.324488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:02.324910 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:02.401088 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] validate_networks() for [('70d105db-93f5-4592-bb80-f58af13e321b', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:25:02.470347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:02.478777 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a748f3a8-e08b-4517-8a8a-d8e4229925b4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.168329 Oct 13 04:25:02.479352 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 323/586] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:02 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:02.583003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:02.773056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:25:02.773480 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:25:02.773953 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:25:02.774394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:25:02.774821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:25:02.784233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:02.784714 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:02.785147 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:02.785863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:02.786339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:02.786760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:02.802809 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:25:02.806704 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:25:02.817366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:25:02.819681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:25:02.865974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ccd2c2c6-dac6-4061-8fbe-89f07f6ccb3a tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.572940 Oct 13 04:25:02.866666 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 264/587] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:25:02 2022] POST /compute/v2.1/servers => generated 402 bytes in 574 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:25:02.878013 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-397aacd0-4732-4d0b-89b1-1bb1676d31a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:02.878878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-397aacd0-4732-4d0b-89b1-1bb1676d31a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:02.932940 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-397aacd0-4732-4d0b-89b1-1bb1676d31a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:02.940156 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-397aacd0-4732-4d0b-89b1-1bb1676d31a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1241 microversion: 2.1 time: 0.065965 Oct 13 04:25:02.940762 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 324/588] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:02 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1241 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:03.505726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:03.506378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:03.513381 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:03.513875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:03.514369 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:03.515071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:03.515528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:03.516496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:03.658773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:03.668750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-57fa57e6-6157-4405-9364-ec6ac30e0254 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.168814 Oct 13 04:25:03.669415 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 265/589] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:03 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:03.963311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:03.963934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:03.971663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:03.972147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:03.972579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:03.973293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:03.973746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:03.974174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:04.009110 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:04.009622 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:04.016699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:04.017583 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:04.018025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:04.018801 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:04.019249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:04.019662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:04.038242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:04.046118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-14c18ab8-2941-4385-b6f7-517c9b0ebb82 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1293 microversion: 2.1 time: 0.086728 Oct 13 04:25:04.046716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 325/590] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:03 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1293 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:04.152427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lazy-loading 'services' on Instance uuid c318cdef-d4ee-46aa-bc4e-91a49cfe1985 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:25:04.270065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7ecb5873-17fa-46d9-b28b-cdab9d6049be tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-volume_attachments/066597f1-58d0-42ea-bc0f-de4e204ffa0c" status: 202 len: 0 microversion: 2.1 time: 0.264649 Oct 13 04:25:04.270684 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 266/591] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:25:04 2022] DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-volume_attachments/066597f1-58d0-42ea-bc0f-de4e204ffa0c => generated 0 bytes in 266 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:25:04.693408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:04.694160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:04.700180 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:04.700726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:04.701219 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:04.701967 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:04.702501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:04.702984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:04.845842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:04.854661 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c8ddc275-0153-4729-b424-08b63b44c831 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.165028 Oct 13 04:25:04.855327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 326/592] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:04 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:05.071085 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:05.072525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:05.079636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:05.080129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:05.080567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:05.081271 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:05.081723 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:05.082320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:05.166047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:05.174610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4c4abd23-6740-4edc-9269-6da6f706e6bf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1299 microversion: 2.1 time: 0.107792 Oct 13 04:25:05.175490 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 267/593] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:05 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1299 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:05.879105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:05.879808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:05.886495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:05.886977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:05.887409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:05.888094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:05.888546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:05.888965 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:05.908569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:06.026897 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:06.036412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-17eb9b8d-edbf-4711-bec9-8bbaa3bf88d4 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.161428 Oct 13 04:25:06.036989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 327/594] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:05 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:06.197081 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:06.197807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:06.205443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:06.206004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:06.206545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:06.207316 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:06.207848 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:06.208343 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:06.393626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:06.401803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ea3996f0-f7fa-46be-afe7-91ca961a4821 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.208643 Oct 13 04:25:06.402402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 268/595] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:06 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:06.423798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-943a12a2-2946-4aa3-9934-68abda4bbf95 req-28939e35-4367-419f-8d53-49d2ffc6afb7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "83b343e2-9707-46ae-a665-5aa2644cb682", "tag": "76e4ebca-6088-4f89-a5c4-07f0634cdbaf"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:06.433973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-943a12a2-2946-4aa3-9934-68abda4bbf95 req-28939e35-4367-419f-8d53-49d2ffc6afb7 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:06.434833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-943a12a2-2946-4aa3-9934-68abda4bbf95 req-28939e35-4367-419f-8d53-49d2ffc6afb7 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:06.435128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-943a12a2-2946-4aa3-9934-68abda4bbf95 req-28939e35-4367-419f-8d53-49d2ffc6afb7 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:06.446688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-943a12a2-2946-4aa3-9934-68abda4bbf95 req-28939e35-4367-419f-8d53-49d2ffc6afb7 service nova] Creating event network-changed:76e4ebca-6088-4f89-a5c4-07f0634cdbaf for instance 83b343e2-9707-46ae-a665-5aa2644cb682 on hv-cinder-86099 Oct 13 04:25:06.451492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-943a12a2-2946-4aa3-9934-68abda4bbf95 req-28939e35-4367-419f-8d53-49d2ffc6afb7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045865 Oct 13 04:25:06.452428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 328/596] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:07.059496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:07.060203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:07.066917 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:07.067405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:07.067837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:07.068529 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:07.068981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:07.069407 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:07.236501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:07.244729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7fcaec43-0a04-4a8f-a545-28c13b9d9d7b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.188974 Oct 13 04:25:07.245404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 269/597] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:07 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:07.426734 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:07.427420 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:07.433426 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:07.433903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:07.434356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:07.435029 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:07.435474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:07.435886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:07.579786 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:07.588163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7a9210f1-de2a-4209-9510-ec17b13c36d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.166400 Oct 13 04:25:07.588737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 329/598] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:07 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:08.268791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:08.269557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:08.270606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:08.271293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:08.276609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:08.276834 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:08.277399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:08.277947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:08.278715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:08.279045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:08.279354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:08.279703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:08.279909 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:08.280471 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:08.280879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:08.281958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:08.282787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:08.423896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:08.432566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d01831dd-28f6-4db0-886a-0053239d971c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1596 microversion: 2.1 time: 0.166861 Oct 13 04:25:08.433400 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 330/599] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:08 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1596 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:08.588090 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:08.600970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2962de04-9f54-4b4f-84c1-f35942b1b63e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.336465 Oct 13 04:25:08.616913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 270/600] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:08 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:08.617279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:08.617279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:08.623538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:08.624443 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:08.626304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:08.627235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:08.627913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:08.628669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:08.771283 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:08.778880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-af1c263d-6eb0-4575-b464-df41c18d06a8 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.169675 Oct 13 04:25:08.779447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 331/601] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:08 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:09.624347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:09.624911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:09.632545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:09.633035 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:09.633461 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:09.634159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:09.634634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:09.635055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:09.733794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e6a1c0b0-f01f-455d-a342-c4cf313bd567 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:09.741127 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e6a1c0b0-f01f-455d-a342-c4cf313bd567 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:09.741678 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e6a1c0b0-f01f-455d-a342-c4cf313bd567 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:09.742172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e6a1c0b0-f01f-455d-a342-c4cf313bd567 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:09.770071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e6a1c0b0-f01f-455d-a342-c4cf313bd567 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Going to try to stop instance {{(pid=118673) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 13 04:25:09.816645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e6a1c0b0-f01f-455d-a342-c4cf313bd567 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action" status: 202 len: 0 microversion: 2.1 time: 0.086860 Oct 13 04:25:09.817258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 332/602] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:25:09 2022] POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:25:09.822466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:09.823165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:09.829585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:09.830064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:09.830519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:09.831218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:09.831663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:09.832083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:09.968012 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:09.976344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c9f618c5-ccc2-4c53-aba5-aae9e9e0c5a3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.157930 Oct 13 04:25:09.976920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 333/603] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:09 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:09.991114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:09.991825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:09.998328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:09.998826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:09.999260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:10.000542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:10.000994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:10.001421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:10.136648 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:10.144838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5ffaac13-6f23-48f3-97d7-c5098bbb6426 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.157513 Oct 13 04:25:10.145410 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 334/604] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:09 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:10.199697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:10.208642 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1ef76e92-1cac-4497-b68a-f951da783e63 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.588133 Oct 13 04:25:10.209227 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 271/605] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:09 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:10.570343 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-9fc78607-8eff-401e-8e2b-dee6d78e28be req-3c0686a2-3b7e-4ca3-891c-73e0ff31d3a9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:10.580710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9fc78607-8eff-401e-8e2b-dee6d78e28be req-3c0686a2-3b7e-4ca3-891c-73e0ff31d3a9 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:10.581339 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9fc78607-8eff-401e-8e2b-dee6d78e28be req-3c0686a2-3b7e-4ca3-891c-73e0ff31d3a9 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:10.581881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9fc78607-8eff-401e-8e2b-dee6d78e28be req-3c0686a2-3b7e-4ca3-891c-73e0ff31d3a9 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:10.595383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-9fc78607-8eff-401e-8e2b-dee6d78e28be req-3c0686a2-3b7e-4ca3-891c-73e0ff31d3a9 service nova] Creating event network-changed:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:25:10.600770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-9fc78607-8eff-401e-8e2b-dee6d78e28be req-3c0686a2-3b7e-4ca3-891c-73e0ff31d3a9 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034415 Oct 13 04:25:10.601885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 335/606] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:10.999488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:11.000734 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:11.013446 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:11.013943 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:11.014406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:11.015121 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:11.015587 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:11.017159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:11.182293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:11.183010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:11.193457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:11.193948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:11.195026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:11.196390 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:11.197054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:11.197858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:11.417025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:11.427244 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-797d7d7c-437d-4b26-866d-fd532b936755 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.431729 Oct 13 04:25:11.442940 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 272/607] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:10 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:11.449359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:11.450195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:11.457514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:11.458177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:11.458989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:11.459868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:11.460467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:11.461028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:11.519492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:11.528103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9aeceb6e-7074-4920-9b79-79d5799084c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.362428 Oct 13 04:25:11.529273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 336/608] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:11 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:11.772817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:11.791711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f084a589-df5e-4f3f-a5a1-89c4e775f038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.347319 Oct 13 04:25:11.792636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 273/609] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:11 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:12.272547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1024143026"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:12.273089 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:12.279481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.280317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.280417 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.281055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.281686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.282152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.411972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.network.security_group_api [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Adding security group 21349ea3-568e-48ec-a7c3-300ddcca47bf to port e493d713-f096-4aa7-b1f4-bc9560bb2da9 Oct 13 04:25:12.454760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:12.455374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:12.462795 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.463508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.464248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.465195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.465810 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.466434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.688612 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:12.697692 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f9c572b6-fb45-4ca0-ba9c-3cfc67ab1f53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.246738 Oct 13 04:25:12.710443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 274/610] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:12 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:12.710443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:12.710443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:12.710786 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.711332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.712031 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.713025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.713814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.714497 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.793969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:12.799573 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f388448e-9d3b-478b-94a4-18c65ae87f4a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/action" status: 202 len: 0 microversion: 2.1 time: 0.529436 Oct 13 04:25:12.803168 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 337/611] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:25:12 2022] POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/action => generated 0 bytes in 532 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:25:12.807012 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-c966ce47-507e-4bc4-8664-6a44a8342726 req-eab2d84d-15e7-4bd5-9ae8-66647e4f2674 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:12.815873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=10 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:12.816906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c966ce47-507e-4bc4-8664-6a44a8342726 req-eab2d84d-15e7-4bd5-9ae8-66647e4f2674 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.817599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c966ce47-507e-4bc4-8664-6a44a8342726 req-eab2d84d-15e7-4bd5-9ae8-66647e4f2674 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.818135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c966ce47-507e-4bc4-8664-6a44a8342726 req-eab2d84d-15e7-4bd5-9ae8-66647e4f2674 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.834779 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-c966ce47-507e-4bc4-8664-6a44a8342726 req-eab2d84d-15e7-4bd5-9ae8-66647e4f2674 service nova] Creating event network-changed:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:25:12.842812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-c966ce47-507e-4bc4-8664-6a44a8342726 req-eab2d84d-15e7-4bd5-9ae8-66647e4f2674 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039124 Oct 13 04:25:12.844802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 338/612] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:12.851098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:12.854966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:12.865012 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.865922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.866694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.867585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.868081 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.868452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.876618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:12.887679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fcb33597-a372-4657-9a2a-2223686ec2b3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.188349 Oct 13 04:25:12.902638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 275/613] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:12 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:12.909883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:12.910747 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:12.918989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.919477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.919901 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.920593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:12.921209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:12.921626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:12.937975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:13.044712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:13.053485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-25b85855-2f87-4ec7-a128-d5a93487cefc tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1643 microversion: 2.1 time: 0.207345 Oct 13 04:25:13.054066 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 339/614] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:12 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1643 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:13.066462 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:13.075139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3280b756-dcb6-418c-a668-b2778a6197f1 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1735 microversion: 2.1 time: 0.171216 Oct 13 04:25:13.075718 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 276/615] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:12 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:13.722314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:13.722945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:13.730136 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:13.730712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:13.731412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:13.732466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:13.732931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:13.733357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:13.888838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:13.897642 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f82d3e93-f060-42ce-b1d7-98a8ddc235f3 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.180444 Oct 13 04:25:13.898643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 340/616] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:13 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:13.916261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:13.917014 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:13.925203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:13.925824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:13.926501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:13.927274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:13.927784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:13.928270 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.099526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:14.100138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:14.106619 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.107115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.107554 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.108318 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.108752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.109702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.146889 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:14.156906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fb9390be-af54-4673-831d-1399a97b3cf5 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.243770 Oct 13 04:25:14.157974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 277/617] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:13 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:14.264053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:14.273081 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e4b315ad-037a-45d3-bf90-b1b8cd34df1e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1715 microversion: 2.1 time: 0.177311 Oct 13 04:25:14.273659 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 341/618] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:14 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1715 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:14.313248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-4a9f5a3b-f272-497a-9c07-f29b9ca01771 req-eef6a5b4-bbf9-45d1-ad48-485d645c4339 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "83b343e2-9707-46ae-a665-5aa2644cb682", "name": "network-vif-plugged", "status": "completed", "tag": "76e4ebca-6088-4f89-a5c4-07f0634cdbaf"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:14.328461 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-4a9f5a3b-f272-497a-9c07-f29b9ca01771 req-eef6a5b4-bbf9-45d1-ad48-485d645c4339 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.328877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-4a9f5a3b-f272-497a-9c07-f29b9ca01771 req-eef6a5b4-bbf9-45d1-ad48-485d645c4339 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.329327 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-4a9f5a3b-f272-497a-9c07-f29b9ca01771 req-eef6a5b4-bbf9-45d1-ad48-485d645c4339 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.340661 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-4a9f5a3b-f272-497a-9c07-f29b9ca01771 req-eef6a5b4-bbf9-45d1-ad48-485d645c4339 service nova] Creating event network-vif-plugged:76e4ebca-6088-4f89-a5c4-07f0634cdbaf for instance 83b343e2-9707-46ae-a665-5aa2644cb682 on hv-cinder-86099 Oct 13 04:25:14.346961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-4a9f5a3b-f272-497a-9c07-f29b9ca01771 req-eef6a5b4-bbf9-45d1-ad48-485d645c4339 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037047 Oct 13 04:25:14.348250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 278/619] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:14.418570 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:14.419065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:14.425901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.426427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.426864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.427562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.428014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.428458 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.518107 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] HTTP exception thrown: Cannot 'detach_volume' instance 168caf0b-e6d8-441d-adfd-821ed4238c11 while it is in vm_state rescued Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Request method failure captured: Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments/4b9f29a3-4a06-4419-ae33-c3c1908cd6ac HTTP/1.1 Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I1Q5IsW5tb9caZYjTnxg76OHJrIRxhA2IsmP-4bjo-Sp8a3KWoY0-TA9ATSJXlQ7ulS9_F3063hs4H1CC9BtmBLHeEh6xOijc0lcEWl3ixN4pgW7TWB892vj9IX631a7RX42s1OkuJTtVHG7SznQ4EGjFdvEIILnlDfDtCnOPDW0s Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:25:14.519601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/0913f8eb06164012b366d62c553d261c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_0913f8eb06164012b366d62c553d261c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: da9c75191a3b4ef49438aae9e35fcf13 Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:25:14.521267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Cannot 'detach_volume' instance 168caf0b-e6d8-441d-adfd-821ed4238c11 while it is in vm_state rescued Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'server_id': '168caf0b-e6d8-441d-adfd-821ed4238c11', 'id': '4b9f29a3-4a06-4419-ae33-c3c1908cd6ac'} Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:25:14.522087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 168caf0b-e6d8-441d-adfd-821ed4238c11 while it is in vm_state rescued Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 168caf0b-e6d8-441d-adfd-821ed4238c11 while it is in vm_state rescued Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:14.523589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Returning 409 to user: Cannot 'detach_volume' instance 168caf0b-e6d8-441d-adfd-821ed4238c11 while it is in vm_state rescued {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:25:14.524730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b76855ef-d6f9-4713-a85d-58fc71d97db5 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments/4b9f29a3-4a06-4419-ae33-c3c1908cd6ac" status: 409 len: 152 microversion: 2.1 time: 0.107223 Oct 13 04:25:14.524730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 342/620] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:25:14 2022] DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments/4b9f29a3-4a06-4419-ae33-c3c1908cd6ac => generated 152 bytes in 109 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 13 04:25:14.532669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:14.533288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:14.539651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.540182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.540659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.541359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.541873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.542417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.599641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:14.609849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-212110df-4aac-4b4c-9df8-414d7bc08f3f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "POST /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/action" status: 202 len: 0 microversion: 2.1 time: 0.081276 Oct 13 04:25:14.610757 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 279/621] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:25:14 2022] POST /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/action => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:25:14.618979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:14.619582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:14.627110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.627609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.628109 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.628904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.629351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.629821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.760413 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:14.769647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c1065ef0-f752-4296-a165-53350c1a5854 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1723 microversion: 2.1 time: 0.154445 Oct 13 04:25:14.770240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 343/622] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:14 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:14.925018 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:14.925627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:14.932132 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.932617 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.933052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:14.933740 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:14.934219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:14.934644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:15.065390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:15.072951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aadb32a9-4fb1-4576-b1b1-35d8869beaa9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.153462 Oct 13 04:25:15.073506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 280/623] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:14 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:15.180869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:15.181489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:15.187759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:15.188606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:15.188984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:15.189676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:15.190125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:15.190569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:15.317261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:15.324792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-14506abe-abd4-4f7e-a53d-881cfb508a3c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.147809 Oct 13 04:25:15.325347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 344/624] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:15 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:15.792991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:15.793863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:15.800269 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:15.800757 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:15.801190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:15.801878 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:15.802359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:15.802784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:15.996660 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:16.004563 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ff056057-8f73-46a7-9464-909cd77481a0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1723 microversion: 2.1 time: 0.215447 Oct 13 04:25:16.005416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 281/625] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:15 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1723 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:16.096057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:16.096670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:16.103674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:16.104181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:16.104801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:16.105516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:16.105956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:16.106669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:16.127080 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:16.242057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:16.249657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-937ea31f-2144-47e4-a785-185b56d4b9d9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1367 microversion: 2.1 time: 0.157976 Oct 13 04:25:16.250938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 345/626] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:16 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:16.347705 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:16.348252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:16.354100 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:16.354611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:16.355044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:16.355737 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:16.356451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:16.356873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:16.487947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:16.495607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f578d117-810b-49c3-89ed-479a1537ef11 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.151971 Oct 13 04:25:16.496179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 282/627] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:16 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:17.028132 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:17.028869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:17.035092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.035661 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.036168 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.036928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.037514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.037971 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.167784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:17.176694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9d693aed-794c-4f22-8ac8-f2f58380afb0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1723 microversion: 2.1 time: 0.152669 Oct 13 04:25:17.177273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 346/628] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:17 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:17.272614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:17.273225 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:17.279686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.280181 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.280593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.281432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.281886 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.282340 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.414774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:17.423494 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4ac43b0f-6910-4eb4-80ff-5e178822b032 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1557 microversion: 2.1 time: 0.154685 Oct 13 04:25:17.424133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 283/629] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:17 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1557 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:17.446069 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:17.446588 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:17.452433 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.453654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.454100 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.454825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.455280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.455707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.520238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:17.520847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:17.526717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.527222 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.527656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.528350 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:17.528808 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:17.529234 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:17.536004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ee56656f-b790-4f6f-bc06-80928c917745 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093703 Oct 13 04:25:17.536602 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 347/630] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:25:17 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:17.660879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:17.668583 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a628173b-f573-44d6-b8dd-79dcabf50355 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.152993 Oct 13 04:25:17.669081 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 284/631] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:17 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:18.200505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:18.201106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:18.207811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:18.208295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:18.208736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:18.209490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:18.209995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:18.210456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:18.399977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:18.410875 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1ef8f65a-7a84-4a2f-9541-4658e07fd525 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1723 microversion: 2.1 time: 0.212092 Oct 13 04:25:18.410875 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 348/632] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:18 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1723 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:18.536914 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-fd23f2bf-a276-423d-ac3f-ef230dd416ee req-8cde82b0-93d2-49a3-9fe9-45f772a4c9d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "83b343e2-9707-46ae-a665-5aa2644cb682", "tag": "76e4ebca-6088-4f89-a5c4-07f0634cdbaf"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:18.547550 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-fd23f2bf-a276-423d-ac3f-ef230dd416ee req-8cde82b0-93d2-49a3-9fe9-45f772a4c9d5 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:18.548323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-fd23f2bf-a276-423d-ac3f-ef230dd416ee req-8cde82b0-93d2-49a3-9fe9-45f772a4c9d5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:18.548816 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-fd23f2bf-a276-423d-ac3f-ef230dd416ee req-8cde82b0-93d2-49a3-9fe9-45f772a4c9d5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:18.560577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-fd23f2bf-a276-423d-ac3f-ef230dd416ee req-8cde82b0-93d2-49a3-9fe9-45f772a4c9d5 service nova] Creating event network-changed:76e4ebca-6088-4f89-a5c4-07f0634cdbaf for instance 83b343e2-9707-46ae-a665-5aa2644cb682 on hv-cinder-86099 Oct 13 04:25:18.566883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-fd23f2bf-a276-423d-ac3f-ef230dd416ee req-8cde82b0-93d2-49a3-9fe9-45f772a4c9d5 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033570 Oct 13 04:25:18.567527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 285/633] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:18.693411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:18.694027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:18.699850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:18.700333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:18.700772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:18.701466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:18.701919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:18.702370 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:18.832642 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:18.840192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6d1d6b57-84de-4470-b2b4-f024b2e49827 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.152495 Oct 13 04:25:18.840768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 349/634] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:18 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:19.433558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:19.434290 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:19.441556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:19.442500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:19.442654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:19.443323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:19.443775 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:19.444274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:19.575003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:19.584229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6c95fdab-49a8-45f2-bfa5-4520a8507480 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1729 microversion: 2.1 time: 0.155709 Oct 13 04:25:19.584802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 286/635] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:19 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1729 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:19.607606 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:19.608096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:19.615210 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:19.615690 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:19.616120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:19.616797 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:19.617240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:19.617654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:19.693586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-093446c2-6e55-4870-a6aa-ef54e8d0287b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090109 Oct 13 04:25:19.694186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 350/636] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:25:19 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:19.865396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:19.866190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:19.872309 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:19.872895 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:19.873395 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:19.874093 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:19.874576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:19.875022 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:20.084505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:20.093581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7e6d7b8b-7cff-473a-a4ac-28112514c898 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.232015 Oct 13 04:25:20.094169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 287/637] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:19 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:25:20.787096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-301e1664-bbaa-457d-94ed-f5e1e497d7b1 req-066d5433-340a-4e77-8092-df3b7a012e19 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "168caf0b-e6d8-441d-adfd-821ed4238c11", "tag": "4f0798be-0676-4fd9-984d-09cbaaf88866"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:20.797586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-301e1664-bbaa-457d-94ed-f5e1e497d7b1 req-066d5433-340a-4e77-8092-df3b7a012e19 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:20.798070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-301e1664-bbaa-457d-94ed-f5e1e497d7b1 req-066d5433-340a-4e77-8092-df3b7a012e19 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:20.798811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-301e1664-bbaa-457d-94ed-f5e1e497d7b1 req-066d5433-340a-4e77-8092-df3b7a012e19 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:20.812759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-301e1664-bbaa-457d-94ed-f5e1e497d7b1 req-066d5433-340a-4e77-8092-df3b7a012e19 service nova] Creating event network-changed:4f0798be-0676-4fd9-984d-09cbaaf88866 for instance 168caf0b-e6d8-441d-adfd-821ed4238c11 on hv-cinder-86099 Oct 13 04:25:20.816202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-301e1664-bbaa-457d-94ed-f5e1e497d7b1 req-066d5433-340a-4e77-8092-df3b7a012e19 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033378 Oct 13 04:25:20.816807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 351/638] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:21.118794 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:21.119552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:21.126667 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:21.127151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:21.127854 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:21.128783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:21.129473 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:21.130259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:21.264893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:21.275059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b8f8fc07-0186-4c56-86f2-38f11ac10192 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1677 microversion: 2.1 time: 0.160135 Oct 13 04:25:21.275741 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 288/639] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:21 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1677 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:22.300807 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:22.301429 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:22.307534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:22.308009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:22.308438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:22.309119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:22.309569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:22.309989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:22.441862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:22.449545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-004a8fad-46a9-409e-9859-66b94fd24880 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1654 microversion: 2.1 time: 0.154038 Oct 13 04:25:22.450104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 352/640] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:22 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1654 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:22.471745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fe01f6ac-9646-4207-9848-086e5d3f872e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:22.478196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fe01f6ac-9646-4207-9848-086e5d3f872e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:22.478696 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fe01f6ac-9646-4207-9848-086e5d3f872e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:22.479126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fe01f6ac-9646-4207-9848-086e5d3f872e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:22.501977 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fe01f6ac-9646-4207-9848-086e5d3f872e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Going to try to start instance {{(pid=118674) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 13 04:25:22.542843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fe01f6ac-9646-4207-9848-086e5d3f872e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action" status: 202 len: 0 microversion: 2.1 time: 0.074906 Oct 13 04:25:22.543457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 289/641] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:25:22 2022] POST /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/action => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:25:22.552336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:22.552879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:22.559201 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:22.559808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:22.560241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:22.560950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:22.561400 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:22.561818 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:22.691633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:22.699518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0f05c8db-df3e-4b7b-99bf-f3632821f7f9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1663 microversion: 2.1 time: 0.151056 Oct 13 04:25:22.700094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 353/642] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:22 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1663 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:22.831644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-540563f1-ced5-4948-adc3-ff1179903dc0 req-c14f1d44-e410-4ab9-b943-ea6d5ac39cfc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "168caf0b-e6d8-441d-adfd-821ed4238c11", "tag": "4f0798be-0676-4fd9-984d-09cbaaf88866"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:22.841266 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-540563f1-ced5-4948-adc3-ff1179903dc0 req-c14f1d44-e410-4ab9-b943-ea6d5ac39cfc service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:22.841837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-540563f1-ced5-4948-adc3-ff1179903dc0 req-c14f1d44-e410-4ab9-b943-ea6d5ac39cfc service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:22.842374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-540563f1-ced5-4948-adc3-ff1179903dc0 req-c14f1d44-e410-4ab9-b943-ea6d5ac39cfc service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:22.853845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-540563f1-ced5-4948-adc3-ff1179903dc0 req-c14f1d44-e410-4ab9-b943-ea6d5ac39cfc service nova] Creating event network-changed:4f0798be-0676-4fd9-984d-09cbaaf88866 for instance 168caf0b-e6d8-441d-adfd-821ed4238c11 on hv-cinder-86099 Oct 13 04:25:22.859796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-540563f1-ced5-4948-adc3-ff1179903dc0 req-c14f1d44-e410-4ab9-b943-ea6d5ac39cfc service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031834 Oct 13 04:25:22.860401 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 290/643] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:23.722598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:23.723210 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:23.729571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:23.730050 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:23.730520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:23.731468 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:23.731920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:23.732346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:23.869736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:23.878268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2797059c-fe1c-4c0b-85b9-c085c26d7675 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1663 microversion: 2.1 time: 0.160209 Oct 13 04:25:23.878832 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 354/644] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:23 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1663 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:24.901836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:24.902586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:24.909727 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:24.910224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:24.910657 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:24.911334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:24.911780 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:24.912197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:25.044047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:25.051911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4c723f65-7390-44da-82ce-7ddfa8d61701 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1667 microversion: 2.1 time: 0.154745 Oct 13 04:25:25.052485 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 291/645] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:24 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1667 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:31.900034 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:31.900590 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:31.903163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=72 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:31.909150 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:31.909654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:31.910087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:31.910811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:31.911258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:31.911674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:31.940749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f4c7f6c8-473b-40e1-b2d4-3df46e43a6f6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments/53d91359-d388-47e5-8291-83967253236f" status: 200 len: 194 microversion: 2.20 time: 0.044785 Oct 13 04:25:31.941310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 355/646] 10.222.0.27 () {60 vars in 1454 bytes} [Thu Oct 13 04:25:31 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments/53d91359-d388-47e5-8291-83967253236f => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:25:32.009126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:32.009643 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:32.016989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:32.017464 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:32.017895 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:32.018603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:32.019051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:32.019467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:32.098800 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lazy-loading 'services' on Instance uuid cc378324-badd-4a63-ba04-4a02fe6551f8 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:25:32.169547 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-80f2f1a3-ce0e-491f-8411-d80015f62701 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments/53d91359-d388-47e5-8291-83967253236f" status: 202 len: 0 microversion: 2.20 time: 0.164115 Oct 13 04:25:32.170394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 292/647] 10.222.0.27 () {60 vars in 1457 bytes} [Thu Oct 13 04:25:32 2022] DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments/53d91359-d388-47e5-8291-83967253236f => generated 0 bytes in 166 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:25:32.179295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:32.182314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:32.188640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:32.189124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:32.189559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:32.190325 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:32.190902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:32.191399 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:32.223952 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d35b47e9-cea7-42b8-94a2-9ad78b069ce7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048632 Oct 13 04:25:32.224517 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 356/648] 10.222.0.27 () {60 vars in 1343 bytes} [Thu Oct 13 04:25:32 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:25:33.238373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:33.240685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:33.243230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:33.250447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:33.251002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:33.251376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:33.252072 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:33.252572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:33.253132 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:33.282982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5bee0d61-261f-480b-a87b-00d353c64edc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.049572 Oct 13 04:25:33.283541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 293/649] 10.222.0.27 () {60 vars in 1343 bytes} [Thu Oct 13 04:25:33 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:33.347744 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:33.348293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:33.354163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:33.354660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:33.355086 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:33.355788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:33.356493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:33.356906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:33.383306 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:25:33.460715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8cde0a12-5ef0-4526-aac4-df2bde0cd14d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 204 len: 0 microversion: 2.20 time: 0.116859 Oct 13 04:25:33.461709 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 357/650] 10.222.0.27 () {60 vars in 1280 bytes} [Thu Oct 13 04:25:33 2022] DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 13 04:25:33.472456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:33.473149 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:33.480639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:33.480788 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:33.481548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:33.482553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:33.483134 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:33.484293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:33.615123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:33.623076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:33.624519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:33.625658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-69449c64-af9c-4783-9360-71edaf7dd313 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.158792 Oct 13 04:25:33.626228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 294/651] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:33 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:34.652040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:34.652771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:34.659773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:34.660255 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:34.660692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:34.661382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:34.661886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:34.662347 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:34.791562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:34.800620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:34.802056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:34.803249 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9c811d08-b530-4bb8-a322-cb6c8904b59a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.155558 Oct 13 04:25:34.803803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 358/652] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:34 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:35.829550 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:35.830144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:35.836731 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:35.837207 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:35.837631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:35.838348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:35.838793 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:35.839210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:36.062482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:36.070724 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:36.072189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:36.073429 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d19837aa-13b6-4a47-a83a-4ecbeca406be tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1723 microversion: 2.20 time: 0.248505 Oct 13 04:25:36.074011 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 295/653] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:35 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1723 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:36.660806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-4d06eb32-7cbc-471b-b773-eba7413a6481 req-71584159-aa26-45d3-b70c-0c865d83fc16 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc378324-badd-4a63-ba04-4a02fe6551f8", "name": "network-vif-deleted", "tag": "5875ba79-ec13-4e89-b04e-d8477ef49e14"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:36.671950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-4d06eb32-7cbc-471b-b773-eba7413a6481 req-71584159-aa26-45d3-b70c-0c865d83fc16 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:36.672432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-4d06eb32-7cbc-471b-b773-eba7413a6481 req-71584159-aa26-45d3-b70c-0c865d83fc16 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:36.672853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-4d06eb32-7cbc-471b-b773-eba7413a6481 req-71584159-aa26-45d3-b70c-0c865d83fc16 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:36.684167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-4d06eb32-7cbc-471b-b773-eba7413a6481 req-71584159-aa26-45d3-b70c-0c865d83fc16 service nova] Creating event network-vif-deleted:5875ba79-ec13-4e89-b04e-d8477ef49e14 for instance cc378324-badd-4a63-ba04-4a02fe6551f8 on hv-cinder-86099 Oct 13 04:25:36.689424 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-4d06eb32-7cbc-471b-b773-eba7413a6481 req-71584159-aa26-45d3-b70c-0c865d83fc16 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032171 Oct 13 04:25:36.690119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 359/654] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:25:37.099295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:37.099909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:37.106734 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:37.107472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:37.108096 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:37.109618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:37.110305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:37.111162 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:37.116441 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:37.145634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:37.146236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:37.146750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:37.147206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lazy-loading 'fault' on Instance uuid cc378324-badd-4a63-ba04-4a02fe6551f8 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:25:37.190805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:37.199048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:37.200511 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:37.201645 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-72a9f20d-cd74-47e1-9542-7d4160eeba9f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 200 len: 1370 microversion: 2.20 time: 0.106260 Oct 13 04:25:37.202344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 296/655] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:37 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 1370 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:38.227802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:38.228494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:38.235538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:38.236011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:38.236438 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:38.237118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:38.237560 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:38.237976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:38.251471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] HTTP exception thrown: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Request method failure captured: Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 HTTP/1.1 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I1cSk8hX3JU_WckuMYN6y60NIhVtGTpF96ZJ00QD5DtLllxQHaJ8R-i05HsPJElOW6QbgdwYhmTtXPrdAxugZ1ndGxeGo8Zr5D2Ef2fcmlG0zmhRrMJz9O1gXDAcU7KNfBOv9tS97zox4jQeP-5hUoCp4lCJTvUnwozSUO8L5hUF4 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Openstack-Nova-Api-Version: 2.20 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:25:38.253057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_75387c76ea384e5c96a336751c613e6e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 35a058c849104a9699ecfcd0c81d398e Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:25:38.254718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'cc378324-badd-4a63-ba04-4a02fe6551f8'} Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:25:38.255520 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:25:38.256925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:25:38.258562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Returning 404 to user: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:25:38.258562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f356fbe2-d7e0-488c-a96b-9d4770e9273b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 404 len: 111 microversion: 2.20 time: 0.031497 Oct 13 04:25:38.258562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 360/656] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:38 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:25:38.263587 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-288e48ef-0b1c-4ec7-bd66-1bd792bc5421 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '.version_select at 0x7f079490fe50>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:38.282244 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-288e48ef-0b1c-4ec7-bd66-1bd792bc5421 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2076485658" status: 204 len: 0 microversion: 2.20 time: 0.022480 Oct 13 04:25:38.282810 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 297/657] 10.222.0.27 () {60 vars in 1262 bytes} [Thu Oct 13 04:25:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2076485658 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 13 04:25:38.913039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-431d94d4-0c55-43b5-8841-aa7caf123b8c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'create', calling method: .version_select at 0x7f079491ed30>, body: {"keypair": {"name": "tempest-keypair-128532449", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIzhDfFpvYMeypN/En2/dIfAyGWQYQ0+q0YPRfqi4iV/UOnnEPHw6ukU1jZ4iX7/uRTZSox9+v33kUvIdYdt1d36r7UUOAcus/I7o++/TeiKeBppu317xr5+11oWore4sw=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:38.928592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-431d94d4-0c55-43b5-8841-aa7caf123b8c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for project 75387c76ea384e5c96a336751c613e6e. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:25:38.932284 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-431d94d4-0c55-43b5-8841-aa7caf123b8c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for user 35a058c849104a9699ecfcd0c81d398e and project 75387c76ea384e5c96a336751c613e6e. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:25:38.954938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-431d94d4-0c55-43b5-8841-aa7caf123b8c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.045688 Oct 13 04:25:38.955514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 361/658] 10.222.0.27 () {62 vars in 1200 bytes} [Thu Oct 13 04:25:38 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 47 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 13 04:25:40.213316 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1323306307", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "73173841-4fa1-43e4-b55e-5df3b82b728b"}], "security_groups": [{"name": "tempest-securitygroup--2087450498"}], "key_name": "tempest-keypair-128532449", "user_data": "}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:40.316176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] validate_networks() for [('73173841-4fa1-43e4-b55e-5df3b82b728b', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:25:40.681409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:25:40.681862 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:25:40.682370 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:25:40.682810 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:25:40.683222 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:25:40.691932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:40.692417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:40.692847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:40.693574 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:40.694027 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:40.694474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:40.710673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for project 75387c76ea384e5c96a336751c613e6e. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:25:40.714571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Getting quotas for user 35a058c849104a9699ecfcd0c81d398e and project 75387c76ea384e5c96a336751c613e6e. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:25:40.725777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:25:40.728089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:25:40.774911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-287b93fc-0680-47e3-945d-bb757fb57f30 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.566418 Oct 13 04:25:40.775557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 298/659] 10.222.0.27 () {62 vars in 1188 bytes} [Thu Oct 13 04:25:40 2022] POST /compute/v2.1/servers => generated 402 bytes in 568 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 13 04:25:40.787833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-81693b6b-039d-478d-8cea-9b84811a4977 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:40.788566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-81693b6b-039d-478d-8cea-9b84811a4977 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:40.844460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81693b6b-039d-478d-8cea-9b84811a4977 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:40.852235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81693b6b-039d-478d-8cea-9b84811a4977 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:40.853696 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81693b6b-039d-478d-8cea-9b84811a4977 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:40.854845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-81693b6b-039d-478d-8cea-9b84811a4977 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1281 microversion: 2.20 time: 0.070982 Oct 13 04:25:40.855394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 362/660] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:40 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1281 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:25:41.881205 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:41.881753 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:41.888518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:41.889338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:41.890530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:41.891229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:41.891689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:41.892118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:41.955594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:41.963760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:41.965232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:41.966422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-11770984-78ec-430f-9e29-0ab4445d348d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1333 microversion: 2.20 time: 0.089625 Oct 13 04:25:41.966988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 299/661] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:41 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1333 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:42.993185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:42.993798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:43.001041 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:43.001762 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:43.002421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:43.003136 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:43.003594 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:43.004022 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:43.150130 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:43.158773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:43.160305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:43.161414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-db2fd371-c2ab-426b-8868-7127a2164455 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.172116 Oct 13 04:25:43.161964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 363/662] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:42 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:43.697558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-d0e3dfe6-c11e-45f1-8840-10411d63fdbb req-16bdecdb-452b-4512-8d01-ff633eb61385 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:43.710152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-d0e3dfe6-c11e-45f1-8840-10411d63fdbb req-16bdecdb-452b-4512-8d01-ff633eb61385 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:43.710936 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-d0e3dfe6-c11e-45f1-8840-10411d63fdbb req-16bdecdb-452b-4512-8d01-ff633eb61385 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:43.711368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-d0e3dfe6-c11e-45f1-8840-10411d63fdbb req-16bdecdb-452b-4512-8d01-ff633eb61385 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:43.722744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-d0e3dfe6-c11e-45f1-8840-10411d63fdbb req-16bdecdb-452b-4512-8d01-ff633eb61385 service nova] Creating event network-changed:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:25:43.728135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-d0e3dfe6-c11e-45f1-8840-10411d63fdbb req-16bdecdb-452b-4512-8d01-ff633eb61385 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034883 Oct 13 04:25:43.728732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 300/663] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:44.187254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:44.187960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:44.194524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:44.195024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:44.195446 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:44.196143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:44.196598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:44.197023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:44.325122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:44.333179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:44.334671 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:44.335783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-136a01a3-82c9-4e29-86ff-adea89ef8486 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.153171 Oct 13 04:25:44.336338 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 364/664] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:44 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:45.216248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:45.219488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:45.226114 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:45.226620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:45.227048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:45.227726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:45.228174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:45.228591 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:45.298179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1c65701e-47ac-4f38-b6c0-4d5c77d7857a tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/action" status: 202 len: 0 microversion: 2.1 time: 0.085932 Oct 13 04:25:45.299052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 301/665] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:25:45 2022] POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:25:45.307030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:45.307667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:45.314140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:45.314645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:45.315076 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:45.315758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:45.316214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:45.316634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:45.320632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:45.363144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:45.363842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:45.371260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:45.371810 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:45.373021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:45.373847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:45.374317 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:45.374978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:45.473294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:45.481888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d57616ad-0051-4d40-a0f7-36d6df0ed877 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1753 microversion: 2.1 time: 0.178687 Oct 13 04:25:45.482707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 365/666] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:45 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1753 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:45.672904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:45.681001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:45.682459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:45.683563 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bae9a54f-0a82-4164-adc4-ff623cf44ada tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.327405 Oct 13 04:25:45.684126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 302/667] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:45 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:25:46.505214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:46.505931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:46.512362 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:46.512838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:46.513263 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:46.513948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:46.514964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:46.515491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:46.664002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:46.672262 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-851d53eb-4171-449d-b149-5c74a6af40da tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1753 microversion: 2.1 time: 0.170940 Oct 13 04:25:46.672826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 366/668] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:46 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1753 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:46.709075 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:46.709702 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:46.715961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:46.716436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:46.716868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:46.717625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:46.718406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:46.719202 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:46.855824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:46.864383 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:46.865818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:46.866948 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5faf8644-8ae5-4560-80e1-4c0410712a49 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.161748 Oct 13 04:25:46.867510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 303/669] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:46 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:47.696207 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:47.696711 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:47.703060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:47.703533 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:47.703956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:47.704629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:47.705068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:47.705480 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:47.848759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:47.857562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c0238f6c-9564-470e-a6c4-6621159b07a9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.165336 Oct 13 04:25:47.858133 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 367/670] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:47 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:47.893071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:47.893676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:47.902130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:47.902648 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:47.903078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:47.903769 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:47.904220 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:47.904642 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:48.035439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:48.043310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:48.044732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:48.045821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-28d63b5f-345a-4cb2-8c0e-2c7d072887fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.157314 Oct 13 04:25:48.046577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 304/671] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:47 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:48.245874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-95503ab8-a260-49f6-8253-e5a65b89a2be req-00d66a3d-8304-463e-958e-3052c383b4c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "name": "network-vif-plugged", "status": "completed", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:48.256275 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-95503ab8-a260-49f6-8253-e5a65b89a2be req-00d66a3d-8304-463e-958e-3052c383b4c4 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:48.256755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-95503ab8-a260-49f6-8253-e5a65b89a2be req-00d66a3d-8304-463e-958e-3052c383b4c4 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:48.257184 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-95503ab8-a260-49f6-8253-e5a65b89a2be req-00d66a3d-8304-463e-958e-3052c383b4c4 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:48.269134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-95503ab8-a260-49f6-8253-e5a65b89a2be req-00d66a3d-8304-463e-958e-3052c383b4c4 service nova] Creating event network-vif-plugged:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:25:48.273971 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-95503ab8-a260-49f6-8253-e5a65b89a2be req-00d66a3d-8304-463e-958e-3052c383b4c4 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031811 Oct 13 04:25:48.274600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 368/672] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:48.881044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:48.881566 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:48.887432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:48.887909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:48.888337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:48.889007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:48.889514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:48.889932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:49.035806 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:49.043999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6abeff5e-a7af-4427-9f92-1d0d603cb51f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.167038 Oct 13 04:25:49.044559 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 305/673] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:48 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:49.071537 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:49.072224 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:49.078673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:49.079148 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:49.079579 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:49.080257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:49.080705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:49.081122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:49.214045 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:49.222882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:49.224309 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:49.225439 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-66d43e0e-4f18-41fe-a961-a52fed5b9508 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.158723 Oct 13 04:25:49.225993 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 369/674] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:49 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:50.067951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:50.068561 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:50.075043 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:50.075528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:50.075964 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:50.076654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:50.077193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:50.077766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:50.220489 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:50.228553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a7d0cb20-5cc0-4479-967d-ad44004cbec5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.164488 Oct 13 04:25:50.229119 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 306/675] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:50 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:50.251739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:50.252365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:50.258613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:50.259116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:50.259542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:50.260227 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:50.260668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:50.261082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:50.389336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:50.397605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:50.399072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:50.400197 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e3ce92a4-9fcd-40f2-b00e-127b5ba28783 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1407 microversion: 2.20 time: 0.153107 Oct 13 04:25:50.400742 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 370/676] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:50 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1407 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:51.253215 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:51.253825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:51.255777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:51.256430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:51.263078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:51.263555 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:51.263984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:51.265209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:51.265652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:51.266066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:51.406704 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:51.415285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a3dcbd1b-203d-4926-9a3e-5720b869dbdb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.166629 Oct 13 04:25:51.415864 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 307/677] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:51 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:51.424863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:51.425471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:51.427484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=6 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:25:51.433046 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:51.433519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:51.433965 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:51.434667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:51.435119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:51.435541 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:51.671937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:51.679879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:51.681307 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:51.682481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4c6c187e-7d76-47f3-b5b0-beb93dae4af5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1599 microversion: 2.20 time: 0.261451 Oct 13 04:25:51.683031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 371/678] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:25:51 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1599 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:25:51.708130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:51.708615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:51.714420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:51.714896 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:51.715320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:51.715989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:51.716428 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:51.716837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:51.850353 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5d8e9711-c61d-4d7b-86f6-e32041974ae2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.145947 Oct 13 04:25:51.850954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 308/679] 10.222.0.27 () {60 vars in 1316 bytes} [Thu Oct 13 04:25:51 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-interface => generated 284 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:25:52.439030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:52.439645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:52.446044 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:52.446558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:52.447000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:52.447700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:52.448154 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:52.448583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:52.593320 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:52.602329 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8bcb4132-91b4-4396-abd8-c12d92f57008 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.167254 Oct 13 04:25:52.602911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 372/680] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:52 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:52.771996 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-0faff915-b618-4f67-b683-4a444466f860 req-88e1643b-a53f-4a29-8555-7e73ee90af73 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:25:52.782530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-0faff915-b618-4f67-b683-4a444466f860 req-88e1643b-a53f-4a29-8555-7e73ee90af73 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:52.783007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-0faff915-b618-4f67-b683-4a444466f860 req-88e1643b-a53f-4a29-8555-7e73ee90af73 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:52.783454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-0faff915-b618-4f67-b683-4a444466f860 req-88e1643b-a53f-4a29-8555-7e73ee90af73 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:52.795222 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-0faff915-b618-4f67-b683-4a444466f860 req-88e1643b-a53f-4a29-8555-7e73ee90af73 service nova] Creating event network-changed:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:25:52.801348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-0faff915-b618-4f67-b683-4a444466f860 req-88e1643b-a53f-4a29-8555-7e73ee90af73 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033451 Oct 13 04:25:52.801952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 309/681] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:25:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:25:53.625884 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:53.626523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:53.633053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:53.633539 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:53.633975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:53.634718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:53.635209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:53.635634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:53.785335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:53.793582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-baecd4b1-869f-4597-9425-ee0ea7cf0a48 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.172090 Oct 13 04:25:53.794183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 373/682] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:53 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:54.819021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:54.819712 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:54.825969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:54.826526 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:54.827007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:54.827731 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:54.828182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:54.828639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:54.972910 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:54.981029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ee4811b3-3703-48f1-9db8-8f30ef6594ba tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.166795 Oct 13 04:25:54.981852 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 310/683] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:54 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:56.005210 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:56.005824 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:56.012361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:56.012845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:56.013280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:56.014651 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:56.015310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:56.015741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:56.160366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:56.168670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dfc7966c-31d8-4e2e-87ca-714ba93b3ec4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.167310 Oct 13 04:25:56.169231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 374/684] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:56 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:57.193032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:57.193638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:57.200165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:57.200644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:57.201077 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:57.202531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:57.203113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:57.203538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:57.343244 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:57.351404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e1cd3e35-ad52-41f3-862b-423f390fd06c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.162240 Oct 13 04:25:57.351972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 311/685] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:57 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:58.375748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:58.376286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:58.382749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:58.383237 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:58.383675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:58.384370 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:58.384825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:58.385251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:58.525521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:58.534250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-71678169-9ef2-437b-af6d-3a185931c76c tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.163148 Oct 13 04:25:58.534828 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 375/686] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:58 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:25:59.558004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:25:59.558654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:25:59.565203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:59.565681 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:59.566113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:59.566867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:25:59.567321 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:25:59.567742 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:25:59.706624 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:25:59.714736 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ca1eea6-0272-4f0e-8999-5b1bc9c6e808 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.160624 Oct 13 04:25:59.715311 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 312/687] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:25:59 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:00.738746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:00.739348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:00.746094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:00.746594 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:00.747019 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:00.747693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:00.748151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:00.748653 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:00.890981 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:00.899035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f701cfda-ccb1-4a6a-b190-5ad54ec418db tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.164123 Oct 13 04:26:00.899599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 376/688] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:00 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:01.735627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:01.736117 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:01.742554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:01.743032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:01.743461 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:01.744144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:01.744597 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:01.745017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:01.824995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'services' on Instance uuid 168caf0b-e6d8-441d-adfd-821ed4238c11 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:01.897203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-34f217aa-7514-4ae2-8d99-9646daa16bf9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments/4b9f29a3-4a06-4419-ae33-c3c1908cd6ac" status: 202 len: 0 microversion: 2.1 time: 0.165625 Oct 13 04:26:01.898029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 313/689] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:26:01 2022] DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments/4b9f29a3-4a06-4419-ae33-c3c1908cd6ac => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:26:01.909444 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:01.911863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:01.914628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:01.920436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:01.921036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:01.921484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:01.922196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:01.922684 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:01.923113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:01.923573 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:01.924261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:01.937595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:01.937998 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:01.938449 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:01.939145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:01.939592 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:01.940007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:01.944167 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:01.959039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-de522b7a-26de-40a5-930b-49d32d5c4802 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056367 Oct 13 04:26:01.963999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 377/690] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:01 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:02.089104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:02.097716 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-82b2ec5b-6981-4fba-a92b-c5aaa7354526 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.178570 Oct 13 04:26:02.098331 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 314/691] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:01 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:02.971958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:02.974160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:02.980227 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:02.980711 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:02.981406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:02.982099 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:02.982595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:02.983571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:03.012478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e5809c1e-83ae-47b6-b7fe-6bb37c2fe9f7 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044497 Oct 13 04:26:03.013030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 378/692] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:02 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:03.121514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:03.122038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:03.129066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:03.129556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:03.129993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:03.130720 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:03.131177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:03.131602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:03.279258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:03.287986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3125edce-4bf3-43db-a892-7faa4a7ba7cd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.170384 Oct 13 04:26:03.288554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 315/693] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:03 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:04.025116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:04.027308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:04.034571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.035006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.035530 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.036120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.036569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.037018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.065853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0eac1bc7-f24b-484c-9463-76d4139cb0af tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044731 Oct 13 04:26:04.066425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 379/694] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:04 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:26:04.148871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:04.150011 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:04.152420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=62,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:04.159104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.159581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.160013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.160703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.161147 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.161566 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.188416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:26:04.259523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-367988a7-6726-4ae6-b263-a39cb988c2ff tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 204 len: 0 microversion: 2.1 time: 0.114335 Oct 13 04:26:04.260094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 316/695] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:26:04 2022] DELETE /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:26:04.268561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:04.269384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:04.275524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.276095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.276684 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.277462 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.277880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.278369 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.311501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:04.312313 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:04.318602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.319639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.319639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.320397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:04.320848 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:04.321304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:04.410765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:04.419154 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a12926f3-dd6a-46cd-b47e-df53406c253f tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1689 microversion: 2.1 time: 0.154876 Oct 13 04:26:04.419737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 380/696] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:04 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1689 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:04.472133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:04.480331 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-82293476-98e3-45f9-9cde-c2077c0a3418 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.172985 Oct 13 04:26:04.480896 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 317/697] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:04 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:05.442296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:05.442902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:05.449202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:05.449678 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:05.450115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:05.450818 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:05.451269 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:05.451686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:05.504333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:05.504935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:05.511390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:05.511869 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:05.512304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:05.512992 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:05.513443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:05.513863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:05.582583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:05.590488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dfc4c67d-db5f-46a3-8c56-f64f69ec2ac0 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1689 microversion: 2.1 time: 0.151990 Oct 13 04:26:05.591063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 381/698] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:05 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1689 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:05.663247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:05.671569 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-13db7d3b-8f62-4520-9ada-8526742b9f29 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.170925 Oct 13 04:26:05.672080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 318/699] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:05 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:06.117310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:06.117916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:06.123834 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:06.125151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:06.125585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:06.126304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:06.126757 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:06.127173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:06.143268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:06.150632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-78d9517c-1f00-4455-9fab-4b50a7a8088f tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/ips" status: 200 len: 150 microversion: 2.1 time: 0.090210 Oct 13 04:26:06.151174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 382/700] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:26:06 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/ips => generated 150 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:06.614341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:06.614923 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:06.621229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:06.621715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:06.622155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:06.622904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:06.623363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:06.623790 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:06.695060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:06.695686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:06.703105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:06.703596 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:06.704026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:06.704736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:06.705182 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:06.705596 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:06.757257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:06.765726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-45c68af7-f89a-49f8-b87a-3783823b417c tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1689 microversion: 2.1 time: 0.155250 Oct 13 04:26:06.780006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 319/701] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:06 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1689 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:06.853146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:06.861603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e847aee4-d412-431f-9467-5f36b0593e8b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.170461 Oct 13 04:26:06.862162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 383/702] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:06 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:07.481740 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "460410c6-8b17-4218-ae88-d9fac552d421"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:07.484203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:07.490179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:07.490695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:07.491133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:07.491817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:07.492273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:07.492697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:07.788977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:07.789612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:07.796620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:07.797101 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:07.797530 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:07.798230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:07.798685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:07.799102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:07.926087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:07.933723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-72987480-4c3e-490a-804e-9dc09c44d234 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1689 microversion: 2.1 time: 0.148573 Oct 13 04:26:07.934503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 384/703] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:07 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1689 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:07.938635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:07.939241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:07.944770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:07.945257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:07.945967 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:07.946931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:07.947391 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:07.947821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:08.091033 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:08.099824 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8ff6119a-2f57-4f18-8e4a-f583548ae611 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.164577 Oct 13 04:26:08.100397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 385/704] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:07 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:08.957692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:08.958427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:08.965131 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:08.965714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:08.967598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:08.967598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:08.967598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:08.967999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:09.020261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0ebcdb90-976b-40a9-a1b0-ddc3b55a7fa4 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.542528 Oct 13 04:26:09.021212 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 320/705] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:26:07 2022] POST /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 194 bytes in 1544 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:09.102915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:09.111655 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-48d846cf-b1c2-4cab-8d4e-e69a9afe693a tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1689 microversion: 2.1 time: 0.157991 Oct 13 04:26:09.112215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 386/706] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:08 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1689 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:09.125484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:09.126297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:09.132496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:09.132976 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:09.133407 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:09.134086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:09.134569 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:09.134993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:09.277747 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:09.286607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5e3c7dec-6bea-469c-b50d-1c1ad9b8d86b tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1758 microversion: 2.1 time: 0.166324 Oct 13 04:26:09.287173 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 321/707] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:09 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1758 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:10.146925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:10.147527 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:10.153853 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:10.154366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:10.154799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:10.155523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:10.155975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:10.156395 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:10.292016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:10.301394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-35caf49e-cd0c-4349-b2c3-e14506d33f9d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1689 microversion: 2.1 time: 0.158175 Oct 13 04:26:10.302091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 387/708] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:10 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1689 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:10.317874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:10.319255 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:10.325581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:10.326077 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:10.326548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:10.327313 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:10.327764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:10.328188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:10.677796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:10.686745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-519e2339-8d0c-4c9e-a2d8-36e108e6d535 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1761 microversion: 2.1 time: 0.372754 Oct 13 04:26:10.687434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 322/709] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:10 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1761 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:11.098454 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-06098d46-8299-4c8d-a75e-e26c88c70ef0 req-cc74773d-073e-40fa-833a-cc16237a0b1c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "168caf0b-e6d8-441d-adfd-821ed4238c11", "name": "network-vif-deleted", "tag": "4f0798be-0676-4fd9-984d-09cbaaf88866"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:11.108918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-06098d46-8299-4c8d-a75e-e26c88c70ef0 req-cc74773d-073e-40fa-833a-cc16237a0b1c service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:11.109404 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-06098d46-8299-4c8d-a75e-e26c88c70ef0 req-cc74773d-073e-40fa-833a-cc16237a0b1c service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:11.110026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-06098d46-8299-4c8d-a75e-e26c88c70ef0 req-cc74773d-073e-40fa-833a-cc16237a0b1c service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:11.122628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-06098d46-8299-4c8d-a75e-e26c88c70ef0 req-cc74773d-073e-40fa-833a-cc16237a0b1c service nova] Creating event network-vif-deleted:4f0798be-0676-4fd9-984d-09cbaaf88866 for instance 168caf0b-e6d8-441d-adfd-821ed4238c11 on hv-cinder-86099 Oct 13 04:26:11.127964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-06098d46-8299-4c8d-a75e-e26c88c70ef0 req-cc74773d-073e-40fa-833a-cc16237a0b1c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033215 Oct 13 04:26:11.128590 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 388/710] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:11.326460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:11.327090 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:11.333546 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:11.334034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:11.334745 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:11.335455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:11.335908 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:11.336500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:11.370535 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:11.371032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:11.371469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:11.371938 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'fault' on Instance uuid 168caf0b-e6d8-441d-adfd-821ed4238c11 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:11.423146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:11.438127 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fcdd309d-fce4-4a37-a2f7-6d782738bf08 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1336 microversion: 2.1 time: 0.115221 Oct 13 04:26:11.439257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 323/711] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:11 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1336 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:12.463805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:12.464544 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:12.471135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:12.471612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:12.472049 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:12.472752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:12.473200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:12.473626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:12.507250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:12.507754 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:12.508192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:12.508644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'fault' on Instance uuid 168caf0b-e6d8-441d-adfd-821ed4238c11 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:12.551193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:12.558049 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6400d980-24cf-42ee-8908-e5dd37f3ad32 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 200 len: 1336 microversion: 2.1 time: 0.099172 Oct 13 04:26:12.558665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 389/712] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:12 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 1336 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:13.635224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:13.635836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 168caf0b-e6d8-441d-adfd-821ed4238c11] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:13.641688 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:13.642983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:13.643580 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:13.644272 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:13.644729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:13.645153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:13.658378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] HTTP exception thrown: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 could not be found. Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Request method failure captured: Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 HTTP/1.1 Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I1Q5IsW5tb9caZYjTnxg76OHJrIRxhA2IsmP-4bjo-Sp8a3KWoY0-TA9ATSJXlQ7ulS9_F3063hs4H1CC9BtmBLHeEh6xOijc0lcEWl3ixN4pgW7TWB892vj9IX631a7RX42s1OkuJTtVHG7SznQ4EGjFdvEIILnlDfDtCnOPDW0s Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:26:13.660017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/0913f8eb06164012b366d62c553d261c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_0913f8eb06164012b366d62c553d261c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: da9c75191a3b4ef49438aae9e35fcf13 Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 could not be found. Oct 13 04:26:13.661572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '168caf0b-e6d8-441d-adfd-821ed4238c11'} Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 could not be found. Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:13.662502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 could not be found. Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 could not be found. Oct 13 04:26:13.663968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:13.665242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Returning 404 to user: Instance 168caf0b-e6d8-441d-adfd-821ed4238c11 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:13.665242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-47ecc487-e03f-44d7-8bfa-5fd87dfcd643 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11" status: 404 len: 111 microversion: 2.1 time: 0.084442 Oct 13 04:26:13.665242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 324/713] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:13 2022] GET /compute/v2.1/servers/168caf0b-e6d8-441d-adfd-821ed4238c11 => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:26:13.670251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-87879f3e-4345-4217-9ffc-8975254dc585 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '.version_select at 0x7f07949fa5e0>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:13.675341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 13 04:26:13.675467 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Thu Oct 13 04:26:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-keypair-1691207034 (ip 10.222.0.27) !!! Oct 13 04:26:13.693256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-87879f3e-4345-4217-9ffc-8975254dc585 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1691207034" status: 202 len: 0 microversion: 2.1 time: 0.026711 Oct 13 04:26:13.693816 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 390/714] 10.222.0.27 () {58 vars in 1221 bytes} [Thu Oct 13 04:26:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1691207034 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:26:16.553079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:16.553572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:16.564082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.564568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.565513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.566198 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.566680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.567153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.579327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "460410c6-8b17-4218-ae88-d9fac552d421"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:16.581621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:16.587511 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.587994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.588433 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.589126 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.589582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.590002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.592644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:26:16.669463 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-496db3cb-e3db-437c-9783-a8b7354c2d16 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "DELETE /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 204 len: 0 microversion: 2.1 time: 0.120324 Oct 13 04:26:16.670178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 325/715] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:26:16 2022] DELETE /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:26:16.678801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Invalid volume: volume 460410c6-8b17-4218-ae88-d9fac552d421 is already attached to instances: 83b343e2-9707-46ae-a665-5aa2644cb682 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: POST /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments HTTP/1.1 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 74 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:26:16.680531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {"volumeAttachment": {"volumeId": "460410c6-8b17-4218-ae88-d9fac552d421"}} Oct 13 04:26:16.682106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Invalid volume: volume 460410c6-8b17-4218-ae88-d9fac552d421 is already attached to instances: 83b343e2-9707-46ae-a665-5aa2644cb682 Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'server_id': '83b343e2-9707-46ae-a665-5aa2644cb682', 'body': {'volumeAttachment': {'volumeId': '460410c6-8b17-4218-ae88-d9fac552d421'}}} Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 460410c6-8b17-4218-ae88-d9fac552d421 is already attached to instances: 83b343e2-9707-46ae-a665-5aa2644cb682 Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:16.682995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 460410c6-8b17-4218-ae88-d9fac552d421 is already attached to instances: 83b343e2-9707-46ae-a665-5aa2644cb682 Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:16.684305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:16.685629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:16.685629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:16.685629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:16.685629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 460410c6-8b17-4218-ae88-d9fac552d421 is already attached to instances: 83b343e2-9707-46ae-a665-5aa2644cb682 Oct 13 04:26:16.685629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:16.685983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:16.686095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 400 to user: Invalid volume: volume 460410c6-8b17-4218-ae88-d9fac552d421 is already attached to instances: 83b343e2-9707-46ae-a665-5aa2644cb682 {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:16.686184 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:16.686612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e374f7fa-4948-44b2-96e0-a8e6f45c99ca tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.107442 Oct 13 04:26:16.686612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 391/716] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:26:16 2022] POST /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 174 bytes in 109 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 13 04:26:16.687627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.688107 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.688540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.689210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.689732 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.690363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.709600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:16.747990 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:16.748498 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:16.755403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.755980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.756493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.757267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.757848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.758311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.832461 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:16.839230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'services' on Instance uuid 83b343e2-9707-46ae-a665-5aa2644cb682 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:16.840240 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-02c81489-f4b1-4780-a113-3973fd3f81a9 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1509 microversion: 2.1 time: 0.163591 Oct 13 04:26:16.840792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 326/717] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:16 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1509 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:16.910038 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b41c145b-633e-4d49-b08e-dfe765766fef tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments/460410c6-8b17-4218-ae88-d9fac552d421" status: 202 len: 0 microversion: 2.1 time: 0.165795 Oct 13 04:26:16.910666 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 392/718] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:26:16 2022] DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments/460410c6-8b17-4218-ae88-d9fac552d421 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:26:16.918965 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:16.921232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:16.928157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.928644 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.929078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.929797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:16.930269 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:16.930689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:16.959551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-69fb2736-5884-4e3a-b523-b68ff08a0f30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044474 Oct 13 04:26:16.960094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 327/719] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:16 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:17.865202 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:17.865896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:17.872122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:17.872609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:17.873042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:17.873727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:17.874178 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:17.874624 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:17.972136 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:17.974368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:17.980652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:17.981397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:17.981837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:17.982555 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:17.983006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:17.983428 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:18.008714 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:18.014340 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6c083376-442c-4f2c-bbc1-0990c2e000d5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046074 Oct 13 04:26:18.014915 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 328/720] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:17 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:18.016501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-77152d70-c683-4935-947d-5a13877592c8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1509 microversion: 2.1 time: 0.156049 Oct 13 04:26:18.017069 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 393/721] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:17 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1509 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:18.698292 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:18.698782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:18.704707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:18.705256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:18.705716 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:18.706531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:18.706989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:18.707411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:18.729285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lazy-loading 'flavor' on Instance uuid f186151e-3b9a-4cdf-8d6e-427fea09de90 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:18.802373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3a29ccee-39f3-4a03-b425-1966b9a2416f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "POST /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90/action" status: 202 len: 0 microversion: 2.1 time: 0.107853 Oct 13 04:26:18.802964 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 329/722] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:26:18 2022] POST /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90/action => generated 0 bytes in 109 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:26:18.811250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:18.811861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:18.818711 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:18.819209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:18.819628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:18.820322 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:18.820770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:18.821190 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:18.959061 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:18.966761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a9cb8773-0969-4883-8852-109986b836b1 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.159317 Oct 13 04:26:18.967319 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 394/723] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:18 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:19.027817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:19.030022 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:19.036879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:19.037514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:19.037949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:19.038708 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:19.039161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:19.039585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:19.040091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:19.040699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:19.046471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:19.046949 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:19.047385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:19.048067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:19.048514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:19.049727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:19.070042 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-97ca7942-d8a9-4b3f-9e76-c319bec1d7c1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046993 Oct 13 04:26:19.070643 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 330/724] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:19 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:19.178094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:19.185631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0caadbd1-e130-4c1a-9eae-4d5fd5e22ea8 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1509 microversion: 2.1 time: 0.149264 Oct 13 04:26:19.186185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 395/725] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:19 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1509 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:19.992870 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:19.993386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:19.999445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:19.999924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.000510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.001078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:20.001604 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.002104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.084669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:20.088274 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:20.096845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:20.097432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.097935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.098806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:20.099335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.099833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.138221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b49ca7ab-de3e-4cca-9cf4-5e7f047d3cf7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059505 Oct 13 04:26:20.145492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 396/726] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:20 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:20.148645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-c4595fce-bfc6-4769-a79c-7937f9327edb req-a32fc0b5-6b8b-41a8-8c18-990da8d90ce7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a3a0563-b11d-4cf6-8e95-0ec0884f227b", "name": "network-vif-deleted", "tag": "d7911061-419d-4200-84cb-0fd55dbfabb4"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:20.162698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c4595fce-bfc6-4769-a79c-7937f9327edb req-a32fc0b5-6b8b-41a8-8c18-990da8d90ce7 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:20.163582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c4595fce-bfc6-4769-a79c-7937f9327edb req-a32fc0b5-6b8b-41a8-8c18-990da8d90ce7 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.164356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c4595fce-bfc6-4769-a79c-7937f9327edb req-a32fc0b5-6b8b-41a8-8c18-990da8d90ce7 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.182919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-c4595fce-bfc6-4769-a79c-7937f9327edb req-a32fc0b5-6b8b-41a8-8c18-990da8d90ce7 service nova] Creating event network-vif-deleted:d7911061-419d-4200-84cb-0fd55dbfabb4 for instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b on hv-cinder-86099 Oct 13 04:26:20.189914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-c4595fce-bfc6-4769-a79c-7937f9327edb req-a32fc0b5-6b8b-41a8-8c18-990da8d90ce7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045351 Oct 13 04:26:20.191325 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 397/727] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:20.214073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:20.215354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:20.223989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:20.224390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:20.225192 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.225880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.226936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:20.227738 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:20.228397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:20.233607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f8b0b5a2-61ed-4146-a577-bab2ed405a94 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.244660 Oct 13 04:26:20.234239 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 331/728] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:19 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:20.294383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:20.302304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-99056aba-0328-4591-a2bc-bf195da4e34d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 200 len: 1293 microversion: 2.1 time: 0.097402 Oct 13 04:26:20.302868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 398/729] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:20 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 1293 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:21.152929 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.155051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.161785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.162288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.162719 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.163399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.163845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.164259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.193838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ed053049-0613-4c92-99a7-a38ec645d317 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044956 Oct 13 04:26:21.194432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 332/730] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:21 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:26:21.257434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.257959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.258387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.258524 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.264286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.264609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.264768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.265095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.265253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.265528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.265897 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.266230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.266373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.266698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.266818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.267120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.292209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:26:21.371525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-903b28be-511f-4820-b898-59f885bc7c81 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 204 len: 0 microversion: 2.1 time: 0.117585 Oct 13 04:26:21.375611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 333/731] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:26:21 2022] DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 13 04:26:21.378486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.379454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 9a3a0563-b11d-4cf6-8e95-0ec0884f227b] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.386274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.386770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.387211 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.387913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.388365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.388791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.403356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] HTTP exception thrown: Instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b could not be found. Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Request method failure captured: Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b HTTP/1.1 Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I1Q5IsW5tb9caZYjTnxg76OHJrIRxhA2IsmP-4bjo-Sp8a3KWoY0-TA9ATSJXlQ7ulS9_F3063hs4H1CC9BtmBLHeEh6xOijc0lcEWl3ixN4pgW7TWB892vj9IX631a7RX42s1OkuJTtVHG7SznQ4EGjFdvEIILnlDfDtCnOPDW0s Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:26:21.404925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/0913f8eb06164012b366d62c553d261c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_0913f8eb06164012b366d62c553d261c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: da9c75191a3b4ef49438aae9e35fcf13 Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b could not be found. Oct 13 04:26:21.406518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '9a3a0563-b11d-4cf6-8e95-0ec0884f227b'} Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b could not be found. Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:21.407396 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b could not be found. Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b could not be found. Oct 13 04:26:21.408785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:21.409845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Returning 404 to user: Instance 9a3a0563-b11d-4cf6-8e95-0ec0884f227b could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:21.415412 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ad6464d5-9ea2-414e-b0df-115e31ecc191 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b" status: 404 len: 111 microversion: 2.1 time: 0.038190 Oct 13 04:26:21.415730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:21.415930 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 334/732] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:21 2022] GET /compute/v2.1/servers/9a3a0563-b11d-4cf6-8e95-0ec0884f227b => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:26:21.417590 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.418235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.424195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.424389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c6ebd7fb-a2b7-45e0-b55a-0d2ee26ea1a5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.171194 Oct 13 04:26:21.424680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.425217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 399/733] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:21 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:21.425391 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.426189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.426879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.427359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.430311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.430798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.436537 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.437030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.437461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.438148 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.438620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.439038 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.464889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:26:21.467966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:21.543737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3ace8cf1-3720-4924-bde9-c64087a3a3ee tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "DELETE /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 204 len: 0 microversion: 2.1 time: 0.117188 Oct 13 04:26:21.544329 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 400/734] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:26:21 2022] DELETE /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:26:21.552454 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:21.553151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:21.558985 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.559476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.559915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.560612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:21.560787 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:21.561071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:21.561494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:21.568832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ea195fb-d244-40c0-8903-a1344efced49 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1682 microversion: 2.1 time: 0.155297 Oct 13 04:26:21.569409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 335/735] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:21 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1682 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:21.690000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:21.697662 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fe624b68-4253-4408-b34e-2e2bc920a03b tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.149203 Oct 13 04:26:21.698242 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 401/736] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:21 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:22.448310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:22.448955 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:22.455379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:22.455858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:22.456287 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:22.457054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:22.457536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:22.457958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:22.590269 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:22.592505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:22.593231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:22.599012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ac4e44fe-2118-401a-ac0c-3fa0ac83fd0f tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.154380 Oct 13 04:26:22.599590 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 336/737] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:22 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:22.600476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:22.600956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:22.601381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:22.602068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:22.602542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:22.602958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:22.721015 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:22.721616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:22.728497 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:22.728975 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:22.729404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:22.730094 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:22.730565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:22.730981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:22.732479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:22.740553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fde5bc21-7b05-4b38-bf3b-04176e1c2468 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1682 microversion: 2.1 time: 0.151848 Oct 13 04:26:22.741124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 402/738] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:22 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1682 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:22.859052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:22.867358 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-dc605013-ce50-4cae-9b34-83532602e0e6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.150271 Oct 13 04:26:22.867933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 337/739] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:22 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:23.623662 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:23.624289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:23.630868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:23.632203 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:23.632641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:23.633326 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:23.633776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:23.634196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:23.762314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:23.763592 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:23.764334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:23.770459 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d3fbe978-5b30-489e-9dfa-7a814f55d096 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.150754 Oct 13 04:26:23.771028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 403/740] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:23 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:23.771219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:23.771711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:23.772152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:23.772886 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:23.773338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:23.773764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:23.889905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:23.890630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:23.897313 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:23.897753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:23.898197 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:23.898910 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:23.899357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:23.899776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:23.905351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:23.913827 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4b24dd85-08d1-4726-a80f-1f347d0dc496 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1682 microversion: 2.1 time: 0.154164 Oct 13 04:26:23.914438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 338/741] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:23 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1682 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:24.029252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:24.037007 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1e109653-ca70-44eb-8d5c-cdb32105a29e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.150985 Oct 13 04:26:24.037571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 404/742] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:23 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:24.794341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:24.794956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:24.801385 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:24.801864 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:24.802352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:24.803036 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:24.803486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:24.803907 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:24.933831 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:24.936697 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:24.937304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:24.942061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bd6b5c7c-7313-4793-b34b-38c405d9a6fc tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.151156 Oct 13 04:26:24.942665 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 339/743] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:24 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:24.944376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:24.944859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:24.945286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:24.945969 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:24.946442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:24.946860 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:25.059792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:25.060333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:25.066720 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:25.067206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:25.067631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:25.068320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:25.069119 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:25.069260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:25.073236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:25.078575 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:25.087062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c4856bb3-3363-4c4c-a461-9b36fd788cc5 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1682 microversion: 2.1 time: 0.154070 Oct 13 04:26:25.087635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 405/744] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:24 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1682 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:25.200215 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:25.208023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-95eb2efb-c8cc-425c-bc7c-f13067837d3e tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.152105 Oct 13 04:26:25.208591 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 340/745] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:25 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:25.965704 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:25.966340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:25.973456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:25.973942 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:25.974411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:25.975122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:25.975580 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:25.975998 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:26.105461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:26.111257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:26.111868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:26.113610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4e2280bc-74ea-4ab2-8bac-d8a53d4f16fb tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.151750 Oct 13 04:26:26.114788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 406/746] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:25 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:26.118769 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:26.119252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:26.119682 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:26.120368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:26.120812 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:26.121228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:26.230632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:26.231430 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:26.237272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:26.237752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:26.238287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:26.238993 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:26.239448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:26.239935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:26.253193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:26.261392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7975124b-b1de-436e-9321-8fb378ca66c7 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1682 microversion: 2.1 time: 0.153929 Oct 13 04:26:26.261956 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 341/747] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:26 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1682 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:26.369096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:26.376766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-bee3268a-b8b7-41a2-9739-63b2f8dfd78d tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.150362 Oct 13 04:26:26.377330 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 407/748] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:26 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:27.137147 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:27.138444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:27.146106 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.146741 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.147235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.148132 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.148582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.149001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.286502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:27.287938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:27.295339 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.296165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.296673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.297453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.298104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.299027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.434502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:27.442386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-05db4cdd-30ae-44a3-b5cf-ad674da37307 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1682 microversion: 2.1 time: 0.161196 Oct 13 04:26:27.446280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 408/749] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:27 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:27.447500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:27.448104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:27.453937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.454522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.455005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.455685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.456146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.456572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.506517 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:27.514669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8e3a30a7-428f-4c49-aab8-0664da35020d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.381497 Oct 13 04:26:27.515556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 342/750] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:27 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:27.594124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:27.602630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fb88aa9c-bca0-4f57-b881-4dc9e7e9efd6 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1521 microversion: 2.1 time: 0.158636 Oct 13 04:26:27.603384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 409/751] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:27 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:27.771025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-ec43b0ce-4beb-4268-bd61-06ba076715bc req-349be7e4-96ed-4e54-822a-e449919220bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "68b9a887-b69f-4e94-b26b-95490a16f5f9", "name": "network-vif-deleted", "tag": "946596f5-0fd5-4db0-a720-f291f93741ef"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:27.780216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ec43b0ce-4beb-4268-bd61-06ba076715bc req-349be7e4-96ed-4e54-822a-e449919220bb service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:27.780694 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ec43b0ce-4beb-4268-bd61-06ba076715bc req-349be7e4-96ed-4e54-822a-e449919220bb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:27.781348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ec43b0ce-4beb-4268-bd61-06ba076715bc req-349be7e4-96ed-4e54-822a-e449919220bb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:27.793962 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-ec43b0ce-4beb-4268-bd61-06ba076715bc req-349be7e4-96ed-4e54-822a-e449919220bb service nova] Creating event network-vif-deleted:946596f5-0fd5-4db0-a720-f291f93741ef for instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 on hv-cinder-86099 Oct 13 04:26:27.799230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-ec43b0ce-4beb-4268-bd61-06ba076715bc req-349be7e4-96ed-4e54-822a-e449919220bb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033807 Oct 13 04:26:27.800128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 343/752] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:28.467748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:28.468710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:28.477104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.477592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.478512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.479517 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.480414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.481603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.522524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.523009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.523434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.524049 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid 83b343e2-9707-46ae-a665-5aa2644cb682 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:28.540631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:28.541322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:28.549351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.550078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.551342 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.552438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.553091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.553771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.576749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:28.586863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b762f235-34dc-422d-969d-6c33554d7a06 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1331 microversion: 2.1 time: 0.123964 Oct 13 04:26:28.587993 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 410/753] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:28 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1331 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:28.593337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:28.595918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:28.613196 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.613710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.614803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.617447 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.617966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.618426 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.653848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0aad61ff-e3b5-403d-882c-7effe5b09a7a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.064203 Oct 13 04:26:28.654793 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 411/754] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:28 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985/os-volume_attachments => generated 25 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 13 04:26:28.659451 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:28.660800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:28.667608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.668328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.668955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.669962 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.670607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.671191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.715348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.715837 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.716456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.717214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lazy-loading 'fault' on Instance uuid 68b9a887-b69f-4e94-b26b-95490a16f5f9 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:28.770825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da9c75191a3b4ef49438aae9e35fcf13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0913f8eb06164012b366d62c553d261c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:28.780320 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1bfcff80-9fae-4513-91db-ed977bbdb038 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 200 len: 1311 microversion: 2.1 time: 0.124353 Oct 13 04:26:28.782353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 412/755] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:28 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 1311 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:28.787813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:28.788523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:28.795700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.796403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.797252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.798129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.798759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.799365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.827108 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:28.831001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:26:28.862517 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bca7534e-2726-4500-927a-687de4de0289 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.326087 Oct 13 04:26:28.879202 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 344/756] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:28 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:28.908936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:28.952282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2ac57acd-9e4d-4619-8426-4238b834bb78 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 204 len: 0 microversion: 2.1 time: 0.168527 Oct 13 04:26:28.952872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 413/757] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:26:28 2022] DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:26:28.961834 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:28.962493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:28.969482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.970339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.971001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:28.971974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:28.972814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:28.973486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.109025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:29.118582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cd14c7e4-d1b9-43de-96d3-d8cf18305f4c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1673 microversion: 2.1 time: 0.160616 Oct 13 04:26:29.119398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 345/758] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:28 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1673 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:29.618937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:29.619545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:29.625889 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.626397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.626826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.627505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.627979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.628411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.658992 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.659463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.660089 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.660772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid 83b343e2-9707-46ae-a665-5aa2644cb682 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:29.702124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:29.709732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-55b3d851-ef8f-4ce7-ab93-5df6929e9d1b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1331 microversion: 2.1 time: 0.095052 Oct 13 04:26:29.710308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 414/759] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:29 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1331 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:29.807523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:29.808131 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] [instance: 68b9a887-b69f-4e94-b26b-95490a16f5f9] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:29.814789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.815406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.815847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.817123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.817570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.817995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.831514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] HTTP exception thrown: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 could not be found. Oct 13 04:26:29.832870 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-7e074563-35ff-48c9-909c-4ff468bf94ee req-447adfe7-77c9-4121-84a7-e3e6ba79dd40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "83b343e2-9707-46ae-a665-5aa2644cb682", "name": "network-vif-deleted", "tag": "76e4ebca-6088-4f89-a5c4-07f0634cdbaf"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Request method failure captured: Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 HTTP/1.1 Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I1Q5IsW5tb9caZYjTnxg76OHJrIRxhA2IsmP-4bjo-Sp8a3KWoY0-TA9ATSJXlQ7ulS9_F3063hs4H1CC9BtmBLHeEh6xOijc0lcEWl3ixN4pgW7TWB892vj9IX631a7RX42s1OkuJTtVHG7SznQ4EGjFdvEIILnlDfDtCnOPDW0s Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:26:29.833460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/0913f8eb06164012b366d62c553d261c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/0913f8eb06164012b366d62c553d261c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_0913f8eb06164012b366d62c553d261c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 0913f8eb06164012b366d62c553d261c Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1136888811 Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: da9c75191a3b4ef49438aae9e35fcf13 Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1136888811-project Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 could not be found. Oct 13 04:26:29.834983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '68b9a887-b69f-4e94-b26b-95490a16f5f9'} Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 could not be found. Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:29.835823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 could not be found. Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 could not be found. Oct 13 04:26:29.837231 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:29.838447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] Returning 404 to user: Instance 68b9a887-b69f-4e94-b26b-95490a16f5f9 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:29.838447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7bb59fb8-e94a-4804-8caf-bfada1a9f9c2 tempest-ServerRescueNegativeTestJSON-1136888811 tempest-ServerRescueNegativeTestJSON-1136888811-project] 10.222.0.27 "GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9" status: 404 len: 111 microversion: 2.1 time: 0.031469 Oct 13 04:26:29.842247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 346/760] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:29 2022] GET /compute/v2.1/servers/68b9a887-b69f-4e94-b26b-95490a16f5f9 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:26:29.842518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7e074563-35ff-48c9-909c-4ff468bf94ee req-447adfe7-77c9-4121-84a7-e3e6ba79dd40 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.842940 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7e074563-35ff-48c9-909c-4ff468bf94ee req-447adfe7-77c9-4121-84a7-e3e6ba79dd40 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.843363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7e074563-35ff-48c9-909c-4ff468bf94ee req-447adfe7-77c9-4121-84a7-e3e6ba79dd40 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.854398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-7e074563-35ff-48c9-909c-4ff468bf94ee req-447adfe7-77c9-4121-84a7-e3e6ba79dd40 service nova] Creating event network-vif-deleted:76e4ebca-6088-4f89-a5c4-07f0634cdbaf for instance 83b343e2-9707-46ae-a665-5aa2644cb682 on hv-cinder-86099 Oct 13 04:26:29.859428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-7e074563-35ff-48c9-909c-4ff468bf94ee req-447adfe7-77c9-4121-84a7-e3e6ba79dd40 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030314 Oct 13 04:26:29.860021 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 415/761] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:29.901516 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:29.902083 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:29.908042 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.908534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.908972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:29.909663 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:29.910118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:29.910570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:30.141996 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:30.142749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:30.144969 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=97,nova_api:DELETE=1,nova_api:UPDATE=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:30.150512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:30.150994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:30.151421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:30.152160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:30.152554 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:30.153033 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:30.159530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:30.168243 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a32136d6-2517-4afc-968b-0da47fcd7f2d tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.271278 Oct 13 04:26:30.182475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 347/762] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:29 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:30.289357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:30.297810 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9a89a3c0-dd0f-4360-9196-2f2a4348b9fa tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1673 microversion: 2.1 time: 0.159958 Oct 13 04:26:30.298612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 416/763] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:30 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1673 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:30.733421 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:30.733981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:30.740043 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:30.740527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:30.740966 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:30.741658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:30.742115 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:30.742568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:30.773758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:30.774279 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:30.774716 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:30.775164 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid 83b343e2-9707-46ae-a665-5aa2644cb682 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:30.854442 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:30.862332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-faa60bb9-1e65-4200-9984-ed5c915d317b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 200 len: 1331 microversion: 2.1 time: 0.133131 Oct 13 04:26:30.862905 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 348/764] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:30 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 1331 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:31.192463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:31.193073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:31.199653 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:31.200133 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:31.200562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:31.201248 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:31.201715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:31.202120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:31.322470 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:31.323467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:31.329859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:31.330718 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:31.331399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:31.332701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:31.333379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:31.334019 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:31.365294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:31.377009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8c209595-3cc3-42e4-9a4e-ea8d1f945927 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.187474 Oct 13 04:26:31.379617 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 417/765] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:31 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:31.615294 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:31.623283 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2de92204-acd2-461a-b389-5724f0ad7b6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1673 microversion: 2.1 time: 0.304649 Oct 13 04:26:31.623860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 349/766] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:31 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1673 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:31.885016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:31.885808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:31.892184 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:31.893016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:31.893551 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:31.894947 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:31.895465 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:31.895983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:31.912025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 HTTP/1.1 Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:26:31.913691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:26:31.915254 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '83b343e2-9707-46ae-a665-5aa2644cb682'} Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:31.916114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:26:31.917643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:31.919101 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:31.919101 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cdb24c07-5940-4547-b09e-e9b3d90f5b6c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 404 len: 111 microversion: 2.1 time: 0.035231 Oct 13 04:26:31.920409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 418/767] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:31 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:26:31.925444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-01cf19ab-a712-43a6-bb82-9283dd2d692e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '.version_select at 0x7f07948a0280>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:31.947949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-01cf19ab-a712-43a6-bb82-9283dd2d692e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-383762571" status: 202 len: 0 microversion: 2.1 time: 0.026232 Oct 13 04:26:31.948941 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 350/768] 10.222.0.27 () {58 vars in 1218 bytes} [Thu Oct 13 04:26:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-383762571 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:26:32.409509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:32.436939 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:32.450890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:32.451663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:32.452529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:32.453379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:32.453937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:32.455143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:32.648668 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:32.649665 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:32.657401 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:32.658473 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:32.659135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:32.660806 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:32.661632 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:32.662350 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:32.700823 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:32.712481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b709ba61-fbc5-4b1f-ac6e-786501f0fb34 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.314654 Oct 13 04:26:32.713880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 419/769] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:32 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:32.839916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-333bdb12-790e-46d9-8a5c-cfd7f64fe223 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: .version_select at 0x7f0794a06160>, body: {"keypair": {"name": "tempest-keypair-1217271909", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOz2l4T+tDmlOGFm1oORwcVGmw8bYzO5BJVRhurJkI5IN83pO2OhIgcDdlvTs8pT0q6uiYlbm2fLbJRntggWGSYnndInFb5P43GPJ1Dx9HkXVzgQTwNH73/6lMBZgQqHag=="}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:32.856642 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-333bdb12-790e-46d9-8a5c-cfd7f64fe223 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:26:32.861069 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-333bdb12-790e-46d9-8a5c-cfd7f64fe223 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'key_pairs'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:26:32.885271 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-333bdb12-790e-46d9-8a5c-cfd7f64fe223 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.049449 Oct 13 04:26:32.886425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 420/770] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:26:32 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:33.300784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:33.309585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1ded6da1-443b-4012-9411-ded7a2eb03cb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1673 microversion: 2.1 time: 0.665872 Oct 13 04:26:33.310993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 351/771] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:32 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1673 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:33.738559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:33.739893 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:33.750794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:33.751295 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:33.754184 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:33.755225 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:33.758327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:33.761053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:33.883909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-b36ff33a-3d39-4262-b7c2-77cc69a45967 req-fa97b172-7dd0-4af5-a0b0-765ab2e8c7d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c318cdef-d4ee-46aa-bc4e-91a49cfe1985", "name": "network-vif-deleted", "tag": "bbd5fde3-e147-43a0-a4cf-beb965d2c6c7"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:33.908009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-b36ff33a-3d39-4262-b7c2-77cc69a45967 req-fa97b172-7dd0-4af5-a0b0-765ab2e8c7d5 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:33.908759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-b36ff33a-3d39-4262-b7c2-77cc69a45967 req-fa97b172-7dd0-4af5-a0b0-765ab2e8c7d5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:33.909051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-b36ff33a-3d39-4262-b7c2-77cc69a45967 req-fa97b172-7dd0-4af5-a0b0-765ab2e8c7d5 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:33.922452 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-b36ff33a-3d39-4262-b7c2-77cc69a45967 req-fa97b172-7dd0-4af5-a0b0-765ab2e8c7d5 service nova] Creating event network-vif-deleted:bbd5fde3-e147-43a0-a4cf-beb965d2c6c7 for instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 on hv-cinder-86099 Oct 13 04:26:33.929152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-b36ff33a-3d39-4262-b7c2-77cc69a45967 req-fa97b172-7dd0-4af5-a0b0-765ab2e8c7d5 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.156167 Oct 13 04:26:33.930365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 352/772] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:34.052102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:34.065055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c1df73c8-5d1e-435c-ba93-e214565a0d74 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.329330 Oct 13 04:26:34.067479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 421/773] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:33 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:34.334987 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:34.336564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:34.342641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:34.350092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:34.352558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:34.353200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:34.359283 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:34.361865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:34.365939 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:34.496635 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:34.515865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1c562ea9-3af6-4a23-a370-83d3741e4e9a tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1605 microversion: 2.1 time: 0.186012 Oct 13 04:26:34.517080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 353/774] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:34 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1605 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:34.593028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1592162262", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "70d105db-93f5-4592-bb80-f58af13e321b"}], "security_groups": [{"name": "tempest-securitygroup--1389429595"}], "key_name": "tempest-keypair-1217271909", "user_data": "}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:35.005054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] validate_networks() for [('70d105db-93f5-4592-bb80-f58af13e321b', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:26:35.089211 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:35.089982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:35.098894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.108270 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.108843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.109715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.110331 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.111262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.389042 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:35.397919 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9ac5700d-c326-4008-a695-1a2fcaf5dffe tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.312979 Oct 13 04:26:35.429146 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 354/775] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:35 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:35.514360 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:35.514905 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:35.524710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.525607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.526260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.527307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.527947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.529182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.562007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ee2da36-f5d8-49fb-b2e5-b9a706aad337 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/ips" status: 200 len: 152 microversion: 2.20 time: 0.131449 Oct 13 04:26:35.563179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 355/776] 10.222.0.27 () {60 vars in 1289 bytes} [Thu Oct 13 04:26:35 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/ips => generated 152 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:26:35.569375 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:35.570304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:35.577393 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.578159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.578849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.579729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.580359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.580962 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.669832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.672662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.673358 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.674187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lazy-loading 'fault' on Instance uuid c318cdef-d4ee-46aa-bc4e-91a49cfe1985 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:35.728903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:26:35.729331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:26:35.729871 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:26:35.730310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:26:35.730872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:26:35.748677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.749519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.750730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.751829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:35.752469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:35.756586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:35.790455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:35.800894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-61983ebb-7786-4bf8-ad16-dbcfbd6022a0 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 200 len: 1327 microversion: 2.1 time: 0.235746 Oct 13 04:26:35.807707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:26:35.813699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Getting quotas for user 999c89e1bc7a426886fc802e28d38596 and project 7397177414dd4ed5baed133cc89e7299. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:26:35.814585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 356/777] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:35 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 1327 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:35.828005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:26:35.832943 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:26:35.917174 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-07249ebe-3f16-438a-9e88-3cb4c6dd9f3e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.332980 Oct 13 04:26:35.918159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 422/778] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:26:34 2022] POST /compute/v2.1/servers => generated 402 bytes in 1337 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:26:35.933031 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-71763043-5974-4fcc-b2a0-d3f9d6932325 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:35.934843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-71763043-5974-4fcc-b2a0-d3f9d6932325 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:36.048741 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-71763043-5974-4fcc-b2a0-d3f9d6932325 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:36.059976 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-71763043-5974-4fcc-b2a0-d3f9d6932325 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1242 microversion: 2.1 time: 0.134670 Oct 13 04:26:36.062475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 357/779] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:35 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1242 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:36.434071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:36.434735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:36.440870 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.441669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.442172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.443088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.443543 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.444558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.618724 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:36.627583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-442117b2-d6b2-4cf7-8716-c7fe40bb9715 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1571 microversion: 2.1 time: 0.198335 Oct 13 04:26:36.628736 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 423/780] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:36 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1571 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:36.654650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:36.655297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:36.664614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.665119 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.665623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.666572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.667326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.667977 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.706898 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118674) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 13 04:26:36.779516 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-743adb2e-d860-44ed-8825-12067e005cf6 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "POST /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90/action" status: 202 len: 0 microversion: 2.1 time: 0.129930 Oct 13 04:26:36.780551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 358/781] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:26:36 2022] POST /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90/action => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:26:36.789997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:36.792187 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:36.799407 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.800134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.800595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.810984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.811513 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.812133 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.826390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:36.827126 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:36.834249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.834945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.835379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.836256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:36.836746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:36.837171 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:36.856548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] HTTP exception thrown: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Request method failure captured: Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 HTTP/1.1 Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I2HLU7FfvJKR98uwPjziNoRi-plQYU_qa31AUawJJfQTDe0ClpEi_AolcXzaGTrMWjpZg5jnnUmsfHEhTDA64zZTOxf6vR_6v7pZrQ5a3ydCzLvSN-EpnkKygAetOOpdbebnBRjk4j66fWOy2g5-ZRtt7NnJKkv9ExzynEwDipG9o Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:26:36.858683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_73d6c8ffc0354e9d889b1abc471602ac", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 1f6cf725908041ccaceb1ffa694ad161 Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:26:36.860431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'c318cdef-d4ee-46aa-bc4e-91a49cfe1985'} Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:36.861165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:26:36.862426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:26:36.863556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Returning 404 to user: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:36.864906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f259330b-4910-4148-b60f-7fe5886ef1b6 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 404 len: 111 microversion: 2.1 time: 0.042049 Oct 13 04:26:36.871159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 359/782] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:36 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:26:36.882820 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5fa4e19c-1479-4d4c-a7c3-1ab963ab5ba7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '.version_select at 0x7f07948ce160>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:36.912407 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5fa4e19c-1479-4d4c-a7c3-1ab963ab5ba7 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-603749871" status: 202 len: 0 microversion: 2.1 time: 0.039711 Oct 13 04:26:36.917760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 360/783] 10.222.0.27 () {58 vars in 1218 bytes} [Thu Oct 13 04:26:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-603749871 => generated 0 bytes in 46 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:26:37.023332 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:37.035989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8979779b-8193-4556-b930-c2aebc01d05b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1519 microversion: 2.1 time: 0.251384 Oct 13 04:26:37.042096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 424/784] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:36 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1519 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:37.088496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:37.089367 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:37.099165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:37.100098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:37.100712 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:37.101583 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:37.102463 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:37.103074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:37.108174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:37.198350 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:37.210386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5dc64b95-b8d3-4f92-a724-0f9e8f96f893 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1294 microversion: 2.1 time: 0.126378 Oct 13 04:26:37.211340 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 361/785] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:37 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1294 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:37.751891 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:37.754028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:37.769621 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:37.771020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:37.771563 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:37.772608 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:37.773159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:37.773716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:37.803006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lazy-loading 'flavor' on Instance uuid 499d8484-0292-40e2-a084-f94dac55a15d {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:38.067767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:38.068394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:38.081154 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.086046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.086545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.087459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.087931 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.088357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.099654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-30280d80-826a-4144-9ae6-b0f3256b0d0d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/action" status: 202 len: 0 microversion: 2.20 time: 0.353412 Oct 13 04:26:38.100664 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 425/786] 10.222.0.27 () {62 vars in 1319 bytes} [Thu Oct 13 04:26:37 2022] POST /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/action => generated 0 bytes in 355 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:26:38.108937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:38.109630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:38.117339 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.117825 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.119546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.120970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.121433 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.121865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.377223 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:38.396542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:38.403585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:38.406169 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c73bdc39-01da-4212-b3f0-9eedb56ac2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.301397 Oct 13 04:26:38.407489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 426/787] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:38 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:26:38.414031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:38.414882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:38.430328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.430982 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.431597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.437440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.441238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.442504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.444356 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:38.452405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a2db2aa6-afe6-46fc-b284-0461cae2e03b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1519 microversion: 2.1 time: 0.390138 Oct 13 04:26:38.459871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 362/788] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:38 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1519 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:38.468710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-89e49176-0b16-426e-b834-8ca017cfca7b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: .version_select at 0x7f07948ce700>, body: {"keypair": {"name": "tempest-keypair-1578838222", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNUugfCCOzHm59+3F3AaUVw0I66kzjP5L60H4zulDWonLnlAZ9HIqzkSz07jTj3yeOJfjzmLe5kYnNGMagUbacdsoLMg2sDtxHh8Ee2jn3KCKZvFv1wvdv/869QF+e4Pjg=="}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:38.510586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-89e49176-0b16-426e-b834-8ca017cfca7b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:26:38.520840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-89e49176-0b16-426e-b834-8ca017cfca7b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for user 1f6cf725908041ccaceb1ffa694ad161 and project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:26:38.588040 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-89e49176-0b16-426e-b834-8ca017cfca7b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.130245 Oct 13 04:26:38.590759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 363/789] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:26:38 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:38.602647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-efb82eab-f40f-4914-8d54-50ff87355ee8 req-e9fb6ed8-3cbd-490a-b0ae-7985c8bb91f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "name": "network-vif-unplugged", "status": "completed", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:38.618792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:38.623373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-efb82eab-f40f-4914-8d54-50ff87355ee8 req-e9fb6ed8-3cbd-490a-b0ae-7985c8bb91f1 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:38.626811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-efb82eab-f40f-4914-8d54-50ff87355ee8 req-e9fb6ed8-3cbd-490a-b0ae-7985c8bb91f1 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:38.630029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-efb82eab-f40f-4914-8d54-50ff87355ee8 req-e9fb6ed8-3cbd-490a-b0ae-7985c8bb91f1 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:38.632045 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5c67b69a-1ee8-4ee2-a381-29f394d6b721 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1300 microversion: 2.1 time: 0.221679 Oct 13 04:26:38.633457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 427/790] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:38 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1300 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:38.646270 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.compute.server_external_events [req-efb82eab-f40f-4914-8d54-50ff87355ee8 req-e9fb6ed8-3cbd-490a-b0ae-7985c8bb91f1 service nova] Unable to find a host for instance f186151e-3b9a-4cdf-8d6e-427fea09de90. Dropping event network-vif-unplugged {{(pid=118674) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 13 04:26:38.647323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-efb82eab-f40f-4914-8d54-50ff87355ee8 req-e9fb6ed8-3cbd-490a-b0ae-7985c8bb91f1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.055624 Oct 13 04:26:38.648235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 364/791] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:38 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 57 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 13 04:26:39.441947 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:39.444794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:39.456939 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:39.457595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:39.458689 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:39.468396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:39.471666 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:39.472394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:39.482453 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:39.484541 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:39.510714 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:39.511434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:39.512233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:39.513079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:39.513621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:39.515349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:39.782555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:39.792809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:39.794745 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:39.796304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9186fd62-0b97-456c-9d4d-14d6d7821311 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.360027 Oct 13 04:26:39.797938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 428/792] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:39 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:39.804148 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:39.805106 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:39.811940 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:39.812756 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:39.813433 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:39.814392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:39.815042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:39.815823 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:39.860743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:39.870900 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ba96c16-d757-4125-a30b-fbf9734d1b44 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1521 microversion: 2.1 time: 0.398787 Oct 13 04:26:39.872125 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 365/793] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:39 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1521 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:40.094152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:40.103331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-eb0a23e8-238d-4be1-9a36-3af64ea89459 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.303809 Oct 13 04:26:40.103911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 429/794] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:39 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:40.742123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-136aebac-6ce6-4c4f-9dba-f389c93791ab req-0874aa10-70cd-4692-b903-dc8e6344b869 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f094e19-297d-4829-8af5-55be12ed8ee2", "tag": "b7fcfa46-fbb2-466a-962d-bdf70352c79c"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:40.754390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-136aebac-6ce6-4c4f-9dba-f389c93791ab req-0874aa10-70cd-4692-b903-dc8e6344b869 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:40.755273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-136aebac-6ce6-4c4f-9dba-f389c93791ab req-0874aa10-70cd-4692-b903-dc8e6344b869 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:40.755870 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-136aebac-6ce6-4c4f-9dba-f389c93791ab req-0874aa10-70cd-4692-b903-dc8e6344b869 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:40.768101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-136aebac-6ce6-4c4f-9dba-f389c93791ab req-0874aa10-70cd-4692-b903-dc8e6344b869 service nova] Creating event network-changed:b7fcfa46-fbb2-466a-962d-bdf70352c79c for instance 4f094e19-297d-4829-8af5-55be12ed8ee2 on hv-cinder-86099 Oct 13 04:26:40.774384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-136aebac-6ce6-4c4f-9dba-f389c93791ab req-0874aa10-70cd-4692-b903-dc8e6344b869 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036013 Oct 13 04:26:40.774994 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 366/795] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:40.811573 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-242626981", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "80a6f545-b088-4d6c-b95e-6cedd0b72a59"}], "security_groups": [{"name": "tempest-securitygroup--505275263"}], "key_name": "tempest-keypair-1578838222", "user_data": "}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:40.826793 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:40.830304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:40.837777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:40.838552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:40.839587 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:40.840469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:40.841089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:40.841519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:40.958146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] validate_networks() for [('80a6f545-b088-4d6c-b95e-6cedd0b72a59', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:26:41.188158 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:41.197817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:41.199415 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:41.200767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9462ff82-9978-44ce-9c51-5043710d7f0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.381361 Oct 13 04:26:41.215492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 367/796] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:40 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:26:41.229923 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:41.230819 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:41.240059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:41.241388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:41.241826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:41.243009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:41.243466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:41.244032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:41.423892 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:26:41.424490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:26:41.425274 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:26:41.426256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:26:41.426819 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:26:41.438597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:41.439194 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:41.439927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:41.441660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:41.442523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:41.443248 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:41.461110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:26:41.465255 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Getting quotas for user 1f6cf725908041ccaceb1ffa694ad161 and project 73d6c8ffc0354e9d889b1abc471602ac. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:26:41.469683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:41.477472 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:26:41.479449 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e6f2e3db-5d9c-4e8b-9c01-6ccd578b781b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.262725 Oct 13 04:26:41.479815 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:26:41.480176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 368/797] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:41 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:41.490987 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:41.492412 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:41.500764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:41.501350 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:41.501813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:41.502683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:41.503852 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:41.504343 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:41.538452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8400bc83-880c-4703-80c1-1f1594fa487f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.726695 Oct 13 04:26:41.539398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 430/798] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:26:40 2022] POST /compute/v2.1/servers => generated 401 bytes in 734 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:26:41.550116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4c3a1756-426a-4eb4-97b0-60287d2263a2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:41.551430 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4c3a1756-426a-4eb4-97b0-60287d2263a2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:41.619968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4c3a1756-426a-4eb4-97b0-60287d2263a2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:41.629592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4c3a1756-426a-4eb4-97b0-60287d2263a2 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1237 microversion: 2.1 time: 0.083204 Oct 13 04:26:41.631288 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 431/799] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:41 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1237 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:41.680513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:41.689739 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-23b9c821-4fdc-42eb-a1f2-bf05545bb376 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.207811 Oct 13 04:26:41.691720 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 369/800] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:41 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:42.233693 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:42.234449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:42.241188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.241783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.242247 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.242942 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.243394 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.243850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.262066 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:42.383077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:42.396553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:42.398074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:42.399267 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e3f4ba54-e149-4061-ac14-362155e0dfca tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.170410 Oct 13 04:26:42.399806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 432/801] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:42 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:42.503460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:42.504155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:42.510580 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.511070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.511503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.512411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.512871 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.513297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.653280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:42.654273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:42.661464 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:42.661695 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.662384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.662838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.663548 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.664000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.664631 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.669651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3e381db4-726f-43f5-83e0-5a632e2aae6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.170014 Oct 13 04:26:42.670304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 370/802] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:42 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:42.713877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:42.714514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:42.720701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.721327 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.722387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.723097 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:42.723577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:42.724171 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:42.731978 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:42.740270 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0fd95b3c-0a30-42d4-8dcc-2dea8fa5d687 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1241 microversion: 2.1 time: 0.091081 Oct 13 04:26:42.741112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 433/803] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:42 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1241 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:42.975074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:42.983187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-eafcf15d-4eaa-4005-b19c-14082a96f23b tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.273233 Oct 13 04:26:42.983762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 371/804] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:42 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:43.424984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:43.426419 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:43.433404 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:43.433980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:43.434509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:43.435440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:43.436117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:43.436775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:43.589185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:43.598677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:43.600253 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:43.601702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-81483563-669f-44ed-a85d-01fac97d5e72 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.180905 Oct 13 04:26:43.614392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 434/805] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:43 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:43.693922 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:43.694554 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:43.702440 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:43.703080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:43.703512 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:43.704205 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:43.704651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:43.705083 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:43.763989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:43.764623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:43.771112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:43.771841 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:43.772594 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:43.773298 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:43.773755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:43.774180 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:43.871856 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:43.880527 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-11e94fcb-8796-462b-816f-1020b45e28ed tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1295 microversion: 2.1 time: 0.120421 Oct 13 04:26:43.881098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 435/806] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:43 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1295 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:43.922089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:43.930227 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0a7f7b7c-68c3-4364-9e16-4e2c86ca6544 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.240531 Oct 13 04:26:43.930755 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 372/807] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:43 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:44.007445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:44.008218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:44.014181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.014703 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.015134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.015833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.016285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.016710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.150790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:44.159042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b3a8e0b6-789f-487a-8ec7-bf3712f4fdcb tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.155501 Oct 13 04:26:44.160512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 436/808] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:44 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:44.628766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:44.629643 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:44.635699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.636182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.636610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.637325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.637770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.638190 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.845759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:44.860952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:44.864502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:44.867629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2c3a5405-9784-4e88-9c4f-25c0914e6bc4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.241780 Oct 13 04:26:44.870235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 373/809] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:44 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:44.904581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:44.906067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:44.913084 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.913927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.914568 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.915453 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.916240 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.916957 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.954458 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:44.955260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:44.968142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.970111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.971466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:44.972529 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:44.975682 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:44.976228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:45.106823 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:45.119005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4b9e3475-5f7a-4235-895f-78cde8f756ee tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.218322 Oct 13 04:26:45.119591 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 437/810] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:44 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:45.183893 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:45.185023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:45.193291 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:45.194300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:45.194926 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:45.196036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:45.196848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:45.197657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:45.235888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:45.244112 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2879df51-1bec-48cf-b884-842a48204dcf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.295028 Oct 13 04:26:45.244692 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 374/811] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:44 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:45.358093 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:45.366780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b12cbe61-c29a-4fb4-afff-98f48de89484 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.187906 Oct 13 04:26:45.367451 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 438/812] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:45 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:45.597255 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-d5eb8816-885e-40d6-9e3d-e982821acb09 req-af531916-2605-4f14-9982-95a9212f7afb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "17727b50-e3a3-48f2-a277-7e2f2ff16903", "tag": "df61582b-5a9b-4646-8d63-4fa70aa7b3ab"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:45.607773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-d5eb8816-885e-40d6-9e3d-e982821acb09 req-af531916-2605-4f14-9982-95a9212f7afb service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:45.608485 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-d5eb8816-885e-40d6-9e3d-e982821acb09 req-af531916-2605-4f14-9982-95a9212f7afb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:45.608914 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-d5eb8816-885e-40d6-9e3d-e982821acb09 req-af531916-2605-4f14-9982-95a9212f7afb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:45.620874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-d5eb8816-885e-40d6-9e3d-e982821acb09 req-af531916-2605-4f14-9982-95a9212f7afb service nova] Creating event network-changed:df61582b-5a9b-4646-8d63-4fa70aa7b3ab for instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 on hv-cinder-86099 Oct 13 04:26:45.625510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-d5eb8816-885e-40d6-9e3d-e982821acb09 req-af531916-2605-4f14-9982-95a9212f7afb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032177 Oct 13 04:26:45.626102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 375/813] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:45.896369 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:45.897004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:45.903305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:45.903795 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:45.904234 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:45.904932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:45.905465 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:45.905915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.033163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:46.041302 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:46.042796 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:46.044316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5574e90b-dbb4-4e92-95ad-584224a832db tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.152311 Oct 13 04:26:46.044881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 439/814] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:45 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:46.144337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:46.144970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:46.151018 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.151523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.151955 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.152636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.153082 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.153500 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.270521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:46.271129 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:46.276995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.277477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.277905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.278628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.279079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.279497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.285625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:46.293173 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-97b44fd2-4aba-4552-be61-87129a754406 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.153959 Oct 13 04:26:46.293740 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 376/815] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:46 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:46.396101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:46.396983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:46.404065 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.404801 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.405588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.407293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.408002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.408649 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.433479 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:46.443845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9ec1d3ad-8bab-4956-ab5e-374ae47fd61c tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.177443 Oct 13 04:26:46.444766 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 440/816] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:46 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:46.573164 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:46.582972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-37676bc9-23cc-447b-8045-b9150ed02cfc tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.197520 Oct 13 04:26:46.583754 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 377/817] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:46 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:46.686470 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-d1d22976-8a86-48df-abf7-3e56efb81c06 req-454d5e6c-d62b-40e5-99b2-fd02a3024332 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f094e19-297d-4829-8af5-55be12ed8ee2", "name": "network-vif-plugged", "status": "completed", "tag": "b7fcfa46-fbb2-466a-962d-bdf70352c79c"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:46.696603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d1d22976-8a86-48df-abf7-3e56efb81c06 req-454d5e6c-d62b-40e5-99b2-fd02a3024332 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:46.697076 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d1d22976-8a86-48df-abf7-3e56efb81c06 req-454d5e6c-d62b-40e5-99b2-fd02a3024332 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:46.697508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-d1d22976-8a86-48df-abf7-3e56efb81c06 req-454d5e6c-d62b-40e5-99b2-fd02a3024332 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:46.708784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-d1d22976-8a86-48df-abf7-3e56efb81c06 req-454d5e6c-d62b-40e5-99b2-fd02a3024332 service nova] Creating event network-vif-plugged:b7fcfa46-fbb2-466a-962d-bdf70352c79c for instance 4f094e19-297d-4829-8af5-55be12ed8ee2 on hv-cinder-86099 Oct 13 04:26:46.713685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-d1d22976-8a86-48df-abf7-3e56efb81c06 req-454d5e6c-d62b-40e5-99b2-fd02a3024332 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.168829 Oct 13 04:26:46.714296 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 441/818] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:47.069918 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:47.070641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:47.076794 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.077275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.077761 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.078478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.078970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.079392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.247376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:47.256615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:47.258045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:47.259241 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2c11a417-2636-4e5c-8cca-234f9b1478cd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.193605 Oct 13 04:26:47.259807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 378/819] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:47 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:47.317599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:47.318317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:47.324412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.324895 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.325331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.326024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.326504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.326983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.457316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:47.465093 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4ec4d06d-2992-42f0-8b9e-77bd7d18b97c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.152635 Oct 13 04:26:47.465657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 442/820] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:47 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:47.468713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:47.469326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:47.476692 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.476692 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.477058 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.477821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.478387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.478831 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.606796 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:47.607405 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:47.608200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:47.614023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.614524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.614960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.615657 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:47.616112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:47.616556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-75bda026-6076-4b30-bd2a-59165fd89501 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.151593 Oct 13 04:26:47.616681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:47.617354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 379/821] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:47 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:47.754088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:47.763680 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4b580b0c-bea4-430d-9456-79b657f00272 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.160378 Oct 13 04:26:47.771741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 443/822] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:47 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:48.285142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:48.285679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:48.292081 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.292567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.293004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.293693 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.294152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.294600 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.448691 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:48.452247 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=100 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:48.458433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:48.459859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:48.461025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-65c0f1bb-29c3-4769-b1c4-66e4a9e9f9e1 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.180088 Oct 13 04:26:48.461773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 380/823] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:48 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:48.488017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:48.488578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:48.491251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:48.496421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.496914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.497353 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.498374 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.498846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.499280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.641325 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:48.649238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-70691f4a-09db-4998-8d9a-ffa62665a37c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.165296 Oct 13 04:26:48.649812 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 444/824] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:48 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:48.697484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:48.698089 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:48.700265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:48.706414 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.706903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.707328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.708016 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.708492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.708913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.796127 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:48.797741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:48.807368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.808008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.808538 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.809338 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:48.809999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:48.810567 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:48.904153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:48.912471 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-62b05cb6-aa0a-4aeb-890f-20ef22c8dd6e tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.218765 Oct 13 04:26:48.913321 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 381/825] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:48 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:48.951164 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:48.960835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7c6b630d-7c72-4da6-a424-55827333d68e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1368 microversion: 2.1 time: 0.169278 Oct 13 04:26:48.961411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 445/826] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:48 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1368 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:49.489137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:49.489743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:49.496961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.497441 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.498080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.498821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.499275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.499699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.648154 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:49.656332 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:49.657772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:49.659484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-23a71ea5-3d96-4943-b833-43dbee4a7500 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.174846 Oct 13 04:26:49.660051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 382/827] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:49 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:49.681160 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:49.682928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:49.692490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.693529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.694122 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.695622 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.696078 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.696688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.829487 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:49.837154 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7c292b2f-0ec5-42da-a6f0-94b544faec6d tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.169270 Oct 13 04:26:49.837720 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 446/828] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:49 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:49.936184 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:49.936726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:49.942891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.943372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.943805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.944486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.944935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.945355 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.983874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:49.984598 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:49.990880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.991373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.991813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:49.992521 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:49.992981 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:49.993414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.080254 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:50.089268 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bbc6e27b-8ffa-4e6f-ad59-715a15157e6b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.156881 Oct 13 04:26:50.089887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 383/829] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:49 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:50.101874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1024143026"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:50.102515 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:50.108768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.109251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.109680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.110398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.110850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.111274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.126550 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:50.134491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-312eb426-9c99-4307-a72a-0bd63ed6b575 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1558 microversion: 2.1 time: 0.154685 Oct 13 04:26:50.135065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 447/830] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:49 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:50.157806 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:50.158335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:50.164855 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.165344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.165782 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.166502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.166985 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.167385 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.241252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.network.security_group_api [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Removing security group 21349ea3-568e-48ec-a7c3-300ddcca47bf from port e493d713-f096-4aa7-b1f4-bc9560bb2da9 Oct 13 04:26:50.246477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-cb72bf4d-dcdb-48d3-9ed1-babe3abf7b84 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092418 Oct 13 04:26:50.247077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 448/831] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:26:50 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:50.573265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-013b2f6b-a0d1-4e9c-9e9a-bb1d034db863 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/action" status: 202 len: 0 microversion: 2.1 time: 0.475164 Oct 13 04:26:50.574388 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 384/832] 10.222.0.27 () {60 vars in 1278 bytes} [Thu Oct 13 04:26:50 2022] POST /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/action => generated 0 bytes in 477 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:26:50.585696 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-8d2952d1-8f5f-42b9-8698-ccb71a698861 req-ff172303-b36f-4c4d-ae6b-ff50717f73bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:50.595589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-8d2952d1-8f5f-42b9-8698-ccb71a698861 req-ff172303-b36f-4c4d-ae6b-ff50717f73bd service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.596125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-8d2952d1-8f5f-42b9-8698-ccb71a698861 req-ff172303-b36f-4c4d-ae6b-ff50717f73bd service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.597829 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-8d2952d1-8f5f-42b9-8698-ccb71a698861 req-ff172303-b36f-4c4d-ae6b-ff50717f73bd service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.614981 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-8d2952d1-8f5f-42b9-8698-ccb71a698861 req-ff172303-b36f-4c4d-ae6b-ff50717f73bd service nova] Creating event network-changed:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:26:50.628725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-8d2952d1-8f5f-42b9-8698-ccb71a698861 req-ff172303-b36f-4c4d-ae6b-ff50717f73bd service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038722 Oct 13 04:26:50.628725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 449/833] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:50.686009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:50.688023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:50.698344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.698912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.699617 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.700527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.701170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.701786 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.865722 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:50.867900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:50.876758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.882985 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.883999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:50.885047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:50.886014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:50.886740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:51.028876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:51.037221 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:51.038695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:51.039863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-09c96799-bea5-4d4a-b1fc-564c69ea68bb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.358786 Oct 13 04:26:51.050098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 385/834] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:50 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:51.114156 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:51.114797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:51.123558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:51.128210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:51.129010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:51.129972 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:51.130675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:51.131315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:51.187726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:51.196765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=100 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:51.200187 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a32189ca-f5ae-4ebf-845a-51d96063c024 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.341395 Oct 13 04:26:51.215335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 450/835] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:50 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:51.495761 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-43d5d159-f161-487b-9356-ead104c86296 req-a50d9c45-798d-4252-942b-7f2a9104f611 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "17727b50-e3a3-48f2-a277-7e2f2ff16903", "name": "network-vif-plugged", "status": "completed", "tag": "df61582b-5a9b-4646-8d63-4fa70aa7b3ab"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:51.500755 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:51.514115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-43d5d159-f161-487b-9356-ead104c86296 req-a50d9c45-798d-4252-942b-7f2a9104f611 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:51.515069 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-43d5d159-f161-487b-9356-ead104c86296 req-a50d9c45-798d-4252-942b-7f2a9104f611 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:51.515803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-43d5d159-f161-487b-9356-ead104c86296 req-a50d9c45-798d-4252-942b-7f2a9104f611 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:51.529890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-43d5d159-f161-487b-9356-ead104c86296 req-a50d9c45-798d-4252-942b-7f2a9104f611 service nova] Creating event network-vif-plugged:df61582b-5a9b-4646-8d63-4fa70aa7b3ab for instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 on hv-cinder-86099 Oct 13 04:26:51.536035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-43d5d159-f161-487b-9356-ead104c86296 req-a50d9c45-798d-4252-942b-7f2a9104f611 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044271 Oct 13 04:26:51.537314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 451/836] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:26:51.559899 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:51.574100 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-93afed60-23b0-40f8-a8b0-b708c1aaf9c9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.463767 Oct 13 04:26:51.576257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 386/837] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:51 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:51.797284 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=18 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:51.859833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-07f8cf6f-0b37-4de3-b069-2c95fd898d50 req-e4cfd2bd-7368-403e-b565-3e51b0922db0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f094e19-297d-4829-8af5-55be12ed8ee2", "tag": "b7fcfa46-fbb2-466a-962d-bdf70352c79c"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:51.871171 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-07f8cf6f-0b37-4de3-b069-2c95fd898d50 req-e4cfd2bd-7368-403e-b565-3e51b0922db0 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:51.872357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-07f8cf6f-0b37-4de3-b069-2c95fd898d50 req-e4cfd2bd-7368-403e-b565-3e51b0922db0 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:51.872964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-07f8cf6f-0b37-4de3-b069-2c95fd898d50 req-e4cfd2bd-7368-403e-b565-3e51b0922db0 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:51.886305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-07f8cf6f-0b37-4de3-b069-2c95fd898d50 req-e4cfd2bd-7368-403e-b565-3e51b0922db0 service nova] Creating event network-changed:b7fcfa46-fbb2-466a-962d-bdf70352c79c for instance 4f094e19-297d-4829-8af5-55be12ed8ee2 on hv-cinder-86099 Oct 13 04:26:51.891978 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-07f8cf6f-0b37-4de3-b069-2c95fd898d50 req-e4cfd2bd-7368-403e-b565-3e51b0922db0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095421 Oct 13 04:26:51.892866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 452/838] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:52.075366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:52.076282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:52.083055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:52.083738 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:52.084284 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:52.085113 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:52.085699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:52.086433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:52.234699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:52.236033 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:52.246753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:52.248047 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:52.248724 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:52.250390 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:52.251205 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:52.251885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:52.440674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:52.449683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a8a70eb6-8e6d-448e-beed-b4b0d21e2184 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1362 microversion: 2.1 time: 0.221372 Oct 13 04:26:52.450280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 453/839] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:52 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1362 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:52.495142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:52.508392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:52.510574 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:52.512337 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6b0769b0-a587-4350-9b01-8fdb3701df84 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.440857 Oct 13 04:26:52.513328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 387/840] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:52 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:52.602420 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:52.603043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:52.620113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:52.620601 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:52.625866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:52.626612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:52.627228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:52.628323 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:52.828949 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:52.837206 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4a2d2c95-771f-46d9-bdae-3513aed692bf tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.241441 Oct 13 04:26:52.850866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 454/841] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:52 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:53.126438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:53.127032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:53.133115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-b74ffc1d-b3a4-49fa-8093-bc865f9a0376 req-62d6781b-a2ae-4a41-bd79-ee8e9b8567b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:53.134118 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.134630 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.135064 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.135769 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.136206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.136625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.144732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b74ffc1d-b3a4-49fa-8093-bc865f9a0376 req-62d6781b-a2ae-4a41-bd79-ee8e9b8567b7 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.145589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b74ffc1d-b3a4-49fa-8093-bc865f9a0376 req-62d6781b-a2ae-4a41-bd79-ee8e9b8567b7 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.146344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b74ffc1d-b3a4-49fa-8093-bc865f9a0376 req-62d6781b-a2ae-4a41-bd79-ee8e9b8567b7 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.159084 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-b74ffc1d-b3a4-49fa-8093-bc865f9a0376 req-62d6781b-a2ae-4a41-bd79-ee8e9b8567b7 service nova] Creating event network-changed:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:26:53.165055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-b74ffc1d-b3a4-49fa-8093-bc865f9a0376 req-62d6781b-a2ae-4a41-bd79-ee8e9b8567b7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037776 Oct 13 04:26:53.166695 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 455/842] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:53.274246 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lazy-loading 'services' on Instance uuid d87c09fa-ebda-45e5-9232-2835de1771a4 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:26:53.356961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-21b14e98-9453-4772-9117-3ef59545dbd9 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments/f3c45e48-4d78-444c-95c8-596a79d73bd5" status: 202 len: 0 microversion: 2.1 time: 0.234142 Oct 13 04:26:53.357562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 388/843] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:26:53 2022] DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments/f3c45e48-4d78-444c-95c8-596a79d73bd5 => generated 0 bytes in 236 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:26:53.367282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:53.369524 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:53.375670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.376146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.376574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.377262 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.379283 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.379740 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.409800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-003ce49b-87ad-4b3c-9559-0269b7d5e3b4 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047203 Oct 13 04:26:53.410718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 456/844] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:53 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:53.474165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:53.475151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:53.481717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.482224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.482674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.483378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.483834 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.484261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.538101 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:53.538879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:53.566305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.566808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.567244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.568163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.568763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.569186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.692538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:53.700635 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7ee9f79b-3789-4e2d-978e-1d549685cb37 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1548 microversion: 2.1 time: 0.231027 Oct 13 04:26:53.714399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 389/845] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:53 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1548 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:53.721626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:53.723983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-057b2655-fe00-42c0-b425-0b4eaf68c116 req-5c772838-fac3-47e1-930e-5e0988f204ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "name": "network-vif-plugged", "status": "completed", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:53.732195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:53.733893 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:53.735114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0bf58b7c-64ac-4663-9939-d39817fc6a7a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.201226 Oct 13 04:26:53.735261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-057b2655-fe00-42c0-b425-0b4eaf68c116 req-5c772838-fac3-47e1-930e-5e0988f204ea service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.745994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 457/846] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:53 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:26:53.745994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:53.745994 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:53.748474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-057b2655-fe00-42c0-b425-0b4eaf68c116 req-5c772838-fac3-47e1-930e-5e0988f204ea service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.748619 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.749039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.749474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.749695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-057b2655-fe00-42c0-b425-0b4eaf68c116 req-5c772838-fac3-47e1-930e-5e0988f204ea service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.750172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.750654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.751077 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.765532 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-057b2655-fe00-42c0-b425-0b4eaf68c116 req-5c772838-fac3-47e1-930e-5e0988f204ea service nova] Creating event network-vif-plugged:9b98fb07-be4c-4add-b232-a8149b2588d6 for instance f186151e-3b9a-4cdf-8d6e-427fea09de90 on hv-cinder-86099 Oct 13 04:26:53.772389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-057b2655-fe00-42c0-b425-0b4eaf68c116 req-5c772838-fac3-47e1-930e-5e0988f204ea service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056396 Oct 13 04:26:53.773066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 390/847] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:53.839417 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f0646602-d584-4a04-a1fe-fbb3db388a3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.101626 Oct 13 04:26:53.847946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 458/848] 10.222.0.27 () {58 vars in 1275 bytes} [Thu Oct 13 04:26:53 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-interface => generated 285 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:53.863487 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:53.864196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:53.870884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.871314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.871823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:53.872536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:53.873124 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:53.873746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:54.250264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:54.260349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-62458862-4d44-4ab6-9638-c452240d9cb8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.401274 Oct 13 04:26:54.261472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 391/849] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:53 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:54.422854 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:54.425849 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:54.432026 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:54.432508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:54.432946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:54.433646 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:54.434107 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:54.434559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:54.464108 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7275fc61-be1c-4a4f-966f-cc27c00ee0bd tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045799 Oct 13 04:26:54.464680 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 459/850] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:54 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:54.760947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:54.761839 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:54.765456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:54.769495 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:54.770235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:54.770833 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:54.771704 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:54.772456 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:54.772890 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.288799 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:55.290015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:55.298701 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.299833 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.300277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.300991 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.301484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.301913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.378841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:55.398850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:55.400577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:55.403669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1af3e551-5df1-4212-9067-b35cf58d5550 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1745 microversion: 2.20 time: 0.646669 Oct 13 04:26:55.404240 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 392/851] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:54 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1745 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:55.466869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:55.477013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:55.477357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8839c738-1a98-4a9d-b9a9-2fb907763885 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.194310 Oct 13 04:26:55.477935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 460/852] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:55 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:55.479782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:55.485894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.486418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.486875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.487571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.488201 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.488626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.518744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7c45254e-64ee-4155-826a-79d9e25103eb tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045096 Oct 13 04:26:55.519341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 393/853] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:26:55 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:26:55.530349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:55.530852 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:55.536894 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.537413 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.537888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.538624 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.539088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.539516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.614746 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] HTTP exception thrown: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 is not attached to volume f3c45e48-4d78-444c-95c8-596a79d73bd5 Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Request method failure captured: Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments/f3c45e48-4d78-444c-95c8-596a79d73bd5 HTTP/1.1 Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5LqUlFnbqvEqupfYBqAfxQWDYk_zXAnelYZfuMyEa5Ftd8JBa14RdjnCPJalDqfSNf16FDpbXTIAulOzhlwaC7j6X52ce-08e3fImLo7hMs8mzj-9H_7q6b7w9emTlGrWEbmV7rXBs9jWDQkvoJKIUHBC49oEZ-gtmIZCLXhFINIng Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f2b13e92741f40bc8044ed4bc999b07f Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestMinimumBasicScenario-1840418566 Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:26:55.623747 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f2b13e92741f40bc8044ed4bc999b07f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f2b13e92741f40bc8044ed4bc999b07f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f2b13e92741f40bc8044ed4bc999b07f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f2b13e92741f40bc8044ed4bc999b07f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestMinimumBasicScenario-1840418566 Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f2b13e92741f40bc8044ed4bc999b07f Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1840418566 Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestMinimumBasicScenario-1840418566-project Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 7582f5c0de6b4b5dae18b0320921e4a2 Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestMinimumBasicScenario-1840418566-project Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:26:55.625382 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 is not attached to volume f3c45e48-4d78-444c-95c8-596a79d73bd5 Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'server_id': 'd87c09fa-ebda-45e5-9232-2835de1771a4', 'id': 'f3c45e48-4d78-444c-95c8-596a79d73bd5'} Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id f3c45e48-4d78-444c-95c8-596a79d73bd5. Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:26:55.626231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 is not attached to volume f3c45e48-4d78-444c-95c8-596a79d73bd5 Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 is not attached to volume f3c45e48-4d78-444c-95c8-596a79d73bd5 Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:26:55.628475 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Returning 404 to user: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 is not attached to volume f3c45e48-4d78-444c-95c8-596a79d73bd5 {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:26:55.629866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fe4b97fe-5b82-44cf-a96b-73ca7cf570ac tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments/f3c45e48-4d78-444c-95c8-596a79d73bd5" status: 404 len: 154 microversion: 2.1 time: 0.099332 Oct 13 04:26:55.629866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 461/854] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:26:55 2022] DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4/os-volume_attachments/f3c45e48-4d78-444c-95c8-596a79d73bd5 => generated 154 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:26:55.713300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-1b68b7d9-212b-43d3-9d1e-92e60c64510d req-7eb17b31-7d4a-402d-8c70-21a822d972c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "17727b50-e3a3-48f2-a277-7e2f2ff16903", "tag": "df61582b-5a9b-4646-8d63-4fa70aa7b3ab"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:55.722481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1b68b7d9-212b-43d3-9d1e-92e60c64510d req-7eb17b31-7d4a-402d-8c70-21a822d972c3 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:55.722964 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1b68b7d9-212b-43d3-9d1e-92e60c64510d req-7eb17b31-7d4a-402d-8c70-21a822d972c3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:55.723399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1b68b7d9-212b-43d3-9d1e-92e60c64510d req-7eb17b31-7d4a-402d-8c70-21a822d972c3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:55.736365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-1b68b7d9-212b-43d3-9d1e-92e60c64510d req-7eb17b31-7d4a-402d-8c70-21a822d972c3 service nova] Creating event network-changed:df61582b-5a9b-4646-8d63-4fa70aa7b3ab for instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 on hv-cinder-86099 Oct 13 04:26:55.743275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-1b68b7d9-212b-43d3-9d1e-92e60c64510d req-7eb17b31-7d4a-402d-8c70-21a822d972c3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033469 Oct 13 04:26:55.744670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 394/855] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:56.433669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:56.434511 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:56.443147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.443763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.444389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.445826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.446485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.447040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.502053 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:56.504593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:56.514365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.514768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.515209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.515915 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.516370 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.517393 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.626702 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:56.638063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:56.640930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:56.642848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9400ac88-05df-463c-8c08-4f41016e020f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1745 microversion: 2.20 time: 0.214057 Oct 13 04:26:56.660155 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 462/856] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:56 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1745 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:56.734968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:56.743576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2f393c5a-821c-45e9-a1a2-c467cdcdeefa tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1577 microversion: 2.1 time: 0.246363 Oct 13 04:26:56.744334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 395/857] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:56 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1577 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:56.782039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:56.782555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:56.789445 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.789937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.790403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.791285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.791737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.792191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.819776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:26:56.904915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ac462b5f-d5cd-412b-9e1c-2b2a8d99f5df tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 204 len: 0 microversion: 2.1 time: 0.126338 Oct 13 04:26:56.910155 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 463/858] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:26:56 2022] DELETE /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:26:56.914379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:56.914984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:56.921729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.922230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.922832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:56.923697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:56.924150 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:56.925300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:57.059265 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-8e10a373-f56a-4f38-9bec-334db812845d req-6c40cf7c-146c-4f91-a8d8-ddefd19241d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "name": "network-vif-plugged", "status": "completed", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:57.072879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:57.079279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-8e10a373-f56a-4f38-9bec-334db812845d req-6c40cf7c-146c-4f91-a8d8-ddefd19241d3 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:57.079780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-8e10a373-f56a-4f38-9bec-334db812845d req-6c40cf7c-146c-4f91-a8d8-ddefd19241d3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:57.080221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-8e10a373-f56a-4f38-9bec-334db812845d req-6c40cf7c-146c-4f91-a8d8-ddefd19241d3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:57.082105 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-417d6e19-4ae4-4788-a972-fc3a598229ee tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1556 microversion: 2.1 time: 0.171529 Oct 13 04:26:57.082813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 396/859] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:56 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1556 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:57.106895 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-8e10a373-f56a-4f38-9bec-334db812845d req-6c40cf7c-146c-4f91-a8d8-ddefd19241d3 service nova] Creating event network-vif-plugged:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:26:57.111828 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-8e10a373-f56a-4f38-9bec-334db812845d req-6c40cf7c-146c-4f91-a8d8-ddefd19241d3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056448 Oct 13 04:26:57.113378 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 464/860] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:26:57.675466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:57.675995 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:57.682403 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:57.682877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:57.683329 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:57.684012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:57.684459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:57.684874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:57.769010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:57.769716 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:57.775902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:57.776384 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:57.776820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:57.777516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:57.777973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:57.778502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:57.819365 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:57.827493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:57.828923 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:57.830122 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-51208df1-7ebb-4187-9621-3f896ea2f81b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1745 microversion: 2.20 time: 0.159261 Oct 13 04:26:57.830765 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 397/861] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:57 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1745 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:57.920648 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:57.930750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-706dc82e-ce9e-46b9-bd82-0d40892152a8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1564 microversion: 2.1 time: 0.166487 Oct 13 04:26:57.931502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 465/862] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:57 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1564 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:58.105120 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:58.105825 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:58.112876 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:58.113410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:58.117133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:58.117133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:58.117133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:58.117133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:58.135335 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:26:58.251319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:58.261185 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d95df8ff-7bce-4a23-a334-17c8aea2de1f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1556 microversion: 2.1 time: 0.159830 Oct 13 04:26:58.261759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 398/863] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:58 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1556 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:26:58.858938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:58.859514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:58.872442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:58.873251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:58.873698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:58.874457 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:58.875464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:58.875958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:59.011975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:59.021314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:59.022804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:59.023975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-eb6a0b98-2264-42d4-8715-91586a50108d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.170624 Oct 13 04:26:59.024517 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 466/864] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:26:58 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:26:59.286123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:26:59.287026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:26:59.293622 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:59.294257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:59.294850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:59.295695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:59.296435 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:59.296889 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:26:59.576712 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:26:59.588701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6c64e2cc-87e7-4362-8aef-74e16ea6b0b2 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1556 microversion: 2.1 time: 0.307605 Oct 13 04:26:59.589288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 399/865] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:26:59 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1556 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:26:59.968879 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-2a59737f-af49-4b17-a0a8-e1e4e5aaa6b1 req-39476f5e-14e8-44b4-a5c6-911231a61947 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d87c09fa-ebda-45e5-9232-2835de1771a4", "name": "network-vif-deleted", "tag": "e493d713-f096-4aa7-b1f4-bc9560bb2da9"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:26:59.984360 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2a59737f-af49-4b17-a0a8-e1e4e5aaa6b1 req-39476f5e-14e8-44b4-a5c6-911231a61947 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:26:59.985203 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2a59737f-af49-4b17-a0a8-e1e4e5aaa6b1 req-39476f5e-14e8-44b4-a5c6-911231a61947 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:26:59.985698 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2a59737f-af49-4b17-a0a8-e1e4e5aaa6b1 req-39476f5e-14e8-44b4-a5c6-911231a61947 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:00.018408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-2a59737f-af49-4b17-a0a8-e1e4e5aaa6b1 req-39476f5e-14e8-44b4-a5c6-911231a61947 service nova] Creating event network-vif-deleted:e493d713-f096-4aa7-b1f4-bc9560bb2da9 for instance d87c09fa-ebda-45e5-9232-2835de1771a4 on hv-cinder-86099 Oct 13 04:27:00.025518 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-2a59737f-af49-4b17-a0a8-e1e4e5aaa6b1 req-39476f5e-14e8-44b4-a5c6-911231a61947 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060906 Oct 13 04:27:00.027069 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 467/866] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:26:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:00.051774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:00.052402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:00.058850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:00.059635 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:00.060236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:00.061110 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:00.061733 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:00.062351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:00.282095 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:00.296862 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:00.299001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:00.301345 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d159afd8-a24d-4fa3-9e6b-57fdf3bed0d5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.255809 Oct 13 04:27:00.302520 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 400/867] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:00 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:00.615419 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:00.616497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:00.623422 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:00.624791 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:00.625771 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:00.626779 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:00.628055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:00.628888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:00.670060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:00.670730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:00.671668 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:00.672228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lazy-loading 'fault' on Instance uuid d87c09fa-ebda-45e5-9232-2835de1771a4 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:27:00.729411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7582f5c0de6b4b5dae18b0320921e4a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2b13e92741f40bc8044ed4bc999b07f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:00.740585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e0afa6e5-6bfe-42b8-83c1-d2a1a4a8f49f tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 200 len: 1349 microversion: 2.1 time: 0.131184 Oct 13 04:27:00.741333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 468/868] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:00 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 1349 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:01.335665 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:01.336911 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:01.344863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:01.345496 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:01.346386 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:01.347535 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:01.348210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:01.348726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:01.522048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:01.531095 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:01.533188 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:01.535281 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e3cf9ecc-d28e-4c63-9029-40f2fc20bef9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.204810 Oct 13 04:27:01.536339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 401/869] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:01 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:27:01.768283 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:01.769637 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] [instance: d87c09fa-ebda-45e5-9232-2835de1771a4] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:01.778094 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:01.779021 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:01.779705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:01.780735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:01.781430 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:01.782091 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:01.789380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:01.812574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] HTTP exception thrown: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 could not be found. Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Request method failure captured: Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 HTTP/1.1 Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5LqUlFnbqvEqupfYBqAfxQWDYk_zXAnelYZfuMyEa5Ftd8JBa14RdjnCPJalDqfSNf16FDpbXTIAulOzhlwaC7j6X52ce-08e3fImLo7hMs8mzj-9H_7q6b7w9emTlGrWEbmV7rXBs9jWDQkvoJKIUHBC49oEZ-gtmIZCLXhFINIng Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: f2b13e92741f40bc8044ed4bc999b07f Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestMinimumBasicScenario-1840418566 Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:27:01.816794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f2b13e92741f40bc8044ed4bc999b07f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/f2b13e92741f40bc8044ed4bc999b07f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/f2b13e92741f40bc8044ed4bc999b07f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_f2b13e92741f40bc8044ed4bc999b07f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestMinimumBasicScenario-1840418566 Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: f2b13e92741f40bc8044ed4bc999b07f Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1840418566 Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestMinimumBasicScenario-1840418566-project Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 7582f5c0de6b4b5dae18b0320921e4a2 Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestMinimumBasicScenario-1840418566-project Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 could not be found. Oct 13 04:27:01.818398 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'd87c09fa-ebda-45e5-9232-2835de1771a4'} Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 could not be found. Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:27:01.819292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 could not be found. Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 could not be found. Oct 13 04:27:01.820715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:27:01.822661 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Returning 404 to user: Instance d87c09fa-ebda-45e5-9232-2835de1771a4 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:27:01.823749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-23b47394-7b3b-41ca-882d-01eb101055d0 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4" status: 404 len: 111 microversion: 2.1 time: 0.060964 Oct 13 04:27:01.828075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 469/870] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:01 2022] GET /compute/v2.1/servers/d87c09fa-ebda-45e5-9232-2835de1771a4 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:27:01.832938 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fd920b63-83e4-4fd5-9002-e4c43009fed5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] Calling method '.version_select at 0x7f0794861b80>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:01.859235 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fd920b63-83e4-4fd5-9002-e4c43009fed5 tempest-TestMinimumBasicScenario-1840418566 tempest-TestMinimumBasicScenario-1840418566-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1798269318" status: 202 len: 0 microversion: 2.1 time: 0.030317 Oct 13 04:27:01.860339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 402/871] 10.222.0.27 () {58 vars in 1272 bytes} [Thu Oct 13 04:27:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1798269318 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:27:02.576226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:02.577198 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:02.599452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:02.601207 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:02.601974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:02.603571 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:02.604565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:02.605894 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:02.854410 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:02.866124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:02.867961 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:02.869927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9ac43e57-778e-4aa4-91fd-f0e3d54cbbc9 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.308424 Oct 13 04:27:02.885380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 470/872] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:02 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:03.909087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:03.910430 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:03.921422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:03.922145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:03.923493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:03.925925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:03.926638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:03.927278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:04.313177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:04.329792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:04.340424 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:04.346174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-350b5eb6-c285-4345-8137-1d55d2d19628 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.442006 Oct 13 04:27:04.365834 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 403/873] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:03 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:05.374948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:05.375711 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:05.388030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:05.389529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:05.390666 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:05.391967 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:05.393235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:05.393979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:05.657063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:05.672515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:05.675576 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:05.677645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d490300e-bf89-49ac-addc-099449a0e528 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.306573 Oct 13 04:27:05.680412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 471/874] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:05 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:06.707840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:06.709616 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:06.717990 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:06.718551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:06.719052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:06.719939 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:06.720411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:06.721368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:07.027017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:07.053559 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:07.055043 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:07.066906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8fb10d08-39ae-4602-bc20-c45b8cc7e2f4 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.356658 Oct 13 04:27:07.067762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 404/875] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:06 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:08.095506 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:08.104110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:08.123383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:08.128995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:08.129820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:08.130792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:08.131436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:08.132124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:08.426052 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:08.437929 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:08.440506 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:08.443927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1f57278a-30a2-4cac-b41d-d8fe66fa0e1d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.354361 Oct 13 04:27:08.455102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 472/876] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:08 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:09.473086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:09.474689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:09.482362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:09.483191 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:09.483887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:09.484970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:09.485650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:09.486841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:09.714073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:09.724767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:09.726267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:09.728204 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-be59252c-1a0f-4a78-8b67-2d9aaf421a5c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.258637 Oct 13 04:27:09.731344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 405/877] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:09 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:10.754176 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:10.754968 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:10.765574 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:10.774016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:10.774901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:10.775909 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:10.776584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:10.777237 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:11.067996 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:11.093562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:11.095376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:11.097268 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a1f03718-3de9-4c19-85a8-03c26b98f481 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.347827 Oct 13 04:27:11.098595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 473/878] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:10 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:12.139362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:12.139362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:12.142564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:12.149445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:12.150041 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:12.150873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:12.152408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:12.155937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:12.426536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:12.436233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:12.438551 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:12.440133 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5dba9b60-eb72-4722-8ca4-38a0f4e1d019 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1740 microversion: 2.20 time: 0.319805 Oct 13 04:27:12.441047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 406/879] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:12 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1740 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:13.473788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:13.475373 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:13.484056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:13.485008 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:13.485628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:13.487578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:13.488236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:13.488885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:13.667639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:13.676092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:13.677556 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:13.678748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-abeceb36-156c-47b0-a7d7-6127a346d51a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1722 microversion: 2.20 time: 0.215564 Oct 13 04:27:13.679290 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 474/880] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:13 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1722 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:14.705641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:14.707014 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:14.713523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:14.714013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:14.714570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:14.715334 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:14.715797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:14.716232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:14.863253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:14.871189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:14.872612 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:14.873888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2e5886d1-61f4-45b0-9513-243a242730fd tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1722 microversion: 2.20 time: 0.172639 Oct 13 04:27:14.874328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 407/881] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:14 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1722 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:15.898908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:15.899519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:15.905860 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:15.906495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:15.906805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:15.907644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:15.908801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:15.909396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.047042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:16.055931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:16.057368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:16.058763 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0c15978d-cf6f-417c-90a2-3f8e8bdf4f1a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1725 microversion: 2.20 time: 0.163801 Oct 13 04:27:16.059313 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 475/882] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:15 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1725 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:16.084783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "db903376-0ff9-4b2a-a177-50bad9da4a37"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:16.087205 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:16.093483 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:16.093969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:16.094443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.095636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:16.095636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:16.096051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.397918 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-99855b58-d39f-45ef-b02a-71f21d1f3331 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.317829 Oct 13 04:27:16.398589 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 408/883] 10.222.0.27 () {62 vars in 1364 bytes} [Thu Oct 13 04:27:16 2022] POST /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-volume_attachments => generated 188 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 13 04:27:16.529145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:16.529722 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:16.536450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:16.537014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:16.537559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.538346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:16.538985 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:16.539481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.808770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-fe92d99c-639d-437e-bab7-03c0a0166823 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-volume_attachments/db903376-0ff9-4b2a-a177-50bad9da4a37" status: 202 len: 0 microversion: 2.20 time: 0.283630 Oct 13 04:27:16.809392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 476/884] 10.222.0.27 () {60 vars in 1457 bytes} [Thu Oct 13 04:27:16 2022] DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-volume_attachments/db903376-0ff9-4b2a-a177-50bad9da4a37 => generated 0 bytes in 285 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:27:16.881286 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:16.881913 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:16.888326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:16.888826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:16.889284 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.889942 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:16.890489 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:16.890847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:16.921419 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118674) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 13 04:27:16.987614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-033b321a-94fe-465d-befb-3e7643f3cfc2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "POST /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/action" status: 202 len: 0 microversion: 2.20 time: 0.110142 Oct 13 04:27:16.988457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 409/885] 10.222.0.27 () {62 vars in 1319 bytes} [Thu Oct 13 04:27:16 2022] POST /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/action => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:27:16.996470 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:16.997164 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:17.004882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:17.005374 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:17.005804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:17.006536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:17.006989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:17.007411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:17.145877 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:17.154145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:17.155632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:17.156835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-94defc50-521e-43c4-bfd5-1643674142dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1673 microversion: 2.20 time: 0.164383 Oct 13 04:27:17.157525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 477/886] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:16 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1673 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:27:17.978803 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-cd4cfd2c-a227-4002-9e2d-92d8cc16d911 req-36dc2868-d2f0-4357-bf05-29375effc369 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "name": "network-vif-unplugged", "status": "completed", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:17.991322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-cd4cfd2c-a227-4002-9e2d-92d8cc16d911 req-36dc2868-d2f0-4357-bf05-29375effc369 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:17.992035 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-cd4cfd2c-a227-4002-9e2d-92d8cc16d911 req-36dc2868-d2f0-4357-bf05-29375effc369 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:17.992458 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-cd4cfd2c-a227-4002-9e2d-92d8cc16d911 req-36dc2868-d2f0-4357-bf05-29375effc369 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:18.006261 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-cd4cfd2c-a227-4002-9e2d-92d8cc16d911 req-36dc2868-d2f0-4357-bf05-29375effc369 service nova] Creating event network-vif-unplugged:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:27:18.014195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-cd4cfd2c-a227-4002-9e2d-92d8cc16d911 req-36dc2868-d2f0-4357-bf05-29375effc369 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042917 Oct 13 04:27:18.015035 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 410/887] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:27:17 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:18.183801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:18.184418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:18.191514 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:18.192009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:18.192448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:18.193146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:18.193629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:18.194058 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:18.334036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:18.342737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:18.344164 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:18.345318 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ad5e1604-0d30-4784-a56b-898f397e8d53 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.166587 Oct 13 04:27:18.346619 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 478/888] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:18 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:19.373383 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:19.374004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:19.381233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:19.381712 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:19.382143 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:19.382846 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:19.383293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:19.383706 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:19.520961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:19.529200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:19.530656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:19.531826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0d7a0765-5bab-41e4-8dfd-d597f5ab0b0c tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.163314 Oct 13 04:27:19.532377 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 411/889] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:19 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:20.558414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:20.559029 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:20.561314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=77,nova_api:UPDATE=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:20.567922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:20.568416 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:20.568842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:20.569542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:20.570004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:20.570440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:20.940899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:20.950467 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:20.952320 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:20.953694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a095b8f2-fe1e-4e37-8cea-91ca283e1107 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.399345 Oct 13 04:27:20.954370 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 479/890] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:20 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:21.980228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:21.982327 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:21.989882 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:21.990548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:21.991055 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:21.991857 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:21.992627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:21.993059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:22.144904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:22.153399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:22.154859 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:22.156012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-82a968e5-0e38-4caa-b110-e13bc01b4f64 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.180215 Oct 13 04:27:22.156570 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 412/891] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:21 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:23.185458 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:23.186954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:23.196925 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:23.197614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:23.197969 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:23.198940 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:23.200273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:23.200904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:23.383478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:23.393627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:23.404342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:23.405582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a6eb4ba8-245e-4d38-8822-7bab49e202a7 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.224552 Oct 13 04:27:23.406558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 480/892] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:23 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:24.434393 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:24.435582 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:24.444683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:24.445556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:24.446239 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:24.447179 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:24.447840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:24.448469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:24.453144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:24.601686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:24.612249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:24.614663 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:24.616380 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-961e6c0e-2271-404d-81a9-f3c97036f5b8 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.185834 Oct 13 04:27:24.617534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 413/893] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:24 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:25.650618 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:25.651178 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:25.658576 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:25.660153 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:25.660875 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:25.661758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:25.663228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:25.663817 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:25.830469 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:25.855491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:25.857535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:25.858963 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-3e2591dd-cc85-417d-bc60-6ec362f54382 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.212327 Oct 13 04:27:25.859528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 481/894] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:25 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:26.885625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:26.886501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:26.889199 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=48,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:26.894588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:26.895339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:26.895850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:26.896664 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:26.897214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:26.897711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:27.038150 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:27.046764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:27.048620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:27.050026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-88a37821-3628-40d2-ab52-70debb9b3fb3 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.168457 Oct 13 04:27:27.052010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 414/895] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:26 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:28.077376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:28.077956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:28.084948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:28.085430 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:28.085869 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:28.086603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:28.087064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:28.087486 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:28.220505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:28.229208 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:28.230690 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:28.232117 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ec683b33-ec61-4ce4-be2b-5d91eda1d84f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.158783 Oct 13 04:27:28.232667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 482/896] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:28 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:29.258328 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:29.258936 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:29.265445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:29.265927 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:29.266390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:29.267083 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:29.267542 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:29.267964 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:29.400963 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:29.414048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:29.416497 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:29.417879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3e15658f-98b9-4b01-8c6b-a2d0e3ce1d36 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.163812 Oct 13 04:27:29.418476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 415/897] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:29 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:27:30.444431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:30.445035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:30.451566 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:30.452055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:30.452488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:30.453179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:30.453630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:30.454055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:30.587356 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:30.595380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:30.596838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:30.597997 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-21654140-de6b-4227-ad0c-fb7d9d08ac34 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.158533 Oct 13 04:27:30.598588 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 483/898] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:30 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:31.624379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:31.624984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:31.631954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:31.632433 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:31.632865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:31.633548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:31.633999 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:31.634454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:31.764514 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:31.772789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:31.774248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:31.775392 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e10a9338-1cfc-4ddd-b236-7d75927bba6e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.154920 Oct 13 04:27:31.775950 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 416/899] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:31 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:32.800991 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:32.801592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:32.808500 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:32.808979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:32.809405 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:32.810090 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:32.810569 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:32.810987 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:32.814927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:32.959079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:32.967542 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:32.969134 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:32.970311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4455412c-366b-42ed-ad7b-5e4848db609a tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.173264 Oct 13 04:27:32.970861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 484/900] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:32 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:27:33.997052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:33.997673 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:34.004216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:34.004710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:34.005151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:34.006037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:34.006527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:34.007004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:34.125735 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-b52f2146-b4ee-4a94-927b-8349ea196f0b req-c25c36af-33a5-41e4-95f5-8bad74118cac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "name": "network-vif-plugged", "status": "completed", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:34.141531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b52f2146-b4ee-4a94-927b-8349ea196f0b req-c25c36af-33a5-41e4-95f5-8bad74118cac service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:34.141921 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b52f2146-b4ee-4a94-927b-8349ea196f0b req-c25c36af-33a5-41e4-95f5-8bad74118cac service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:34.142613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b52f2146-b4ee-4a94-927b-8349ea196f0b req-c25c36af-33a5-41e4-95f5-8bad74118cac service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:34.157605 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-b52f2146-b4ee-4a94-927b-8349ea196f0b req-c25c36af-33a5-41e4-95f5-8bad74118cac service nova] Creating event network-vif-plugged:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:27:34.166191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-b52f2146-b4ee-4a94-927b-8349ea196f0b req-c25c36af-33a5-41e4-95f5-8bad74118cac service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043073 Oct 13 04:27:34.167367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 485/901] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:27:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:34.168757 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:34.181675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:34.183078 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:34.184249 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-080945bd-ed02-4769-b082-080baed80a91 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.191077 Oct 13 04:27:34.184813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 417/902] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:33 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:35.212452 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:35.213138 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:35.219314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:35.219867 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:35.220300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:35.220991 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:35.221428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:35.221846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:35.355527 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:35.363975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:35.365440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:35.366726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-130dc7c0-681b-4cbf-a517-b89d1ae51bfb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.159554 Oct 13 04:27:35.367300 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 486/903] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:35 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:36.392762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:36.393487 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:36.401890 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:36.402416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:36.402849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:36.403553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:36.404001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:36.404432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:36.560805 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:36.570383 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:36.572128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:36.573285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-bb1aa7cc-02aa-48d8-b410-919ef77ca63b tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1731 microversion: 2.20 time: 0.184923 Oct 13 04:27:36.573845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 418/904] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:36 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1731 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:27:36.841803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:36.842344 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:36.848976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:36.849460 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:36.849904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:36.851128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:36.851784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:36.852208 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:36.875641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-863925b4-10b3-4cb3-9039-a846335fd681 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/ips" status: 200 len: 150 microversion: 2.1 time: 0.037805 Oct 13 04:27:36.876239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 487/905] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:27:36 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/ips => generated 150 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:37.598700 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:37.599293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:37.605584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:37.606347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:37.606781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:37.607459 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:37.607903 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:37.608320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:37.733336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:37.741920 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:37.743423 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:37.744567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-275127e3-8a84-42ac-9e2e-29e3bb8a5ca6 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1718 microversion: 2.20 time: 0.149823 Oct 13 04:27:37.745127 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 419/906] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:27:37 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1718 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 13 04:27:38.214463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9aa52298-602c-481e-ae1b-4293f54da6da"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:38.216888 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:38.223615 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:38.224114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:38.224522 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:38.225232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:38.225748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:38.226311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:38.504367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f9c399ca-6028-407b-964f-4fc260cc7123 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "POST /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.294450 Oct 13 04:27:38.504982 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 488/907] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:27:38 2022] POST /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 194 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:39.655128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:39.655621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:39.662623 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:39.663129 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:39.664054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:39.664762 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:39.665208 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:39.665627 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:39.689222 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7809ae2f-2463-4b4a-8292-07f8cd22e163 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/ips" status: 200 len: 146 microversion: 2.1 time: 0.038005 Oct 13 04:27:39.689783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 420/908] 10.222.0.27 () {58 vars in 1248 bytes} [Thu Oct 13 04:27:39 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/ips => generated 146 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:41.018097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4dabbe27-59a3-4a93-a5a0-7c1dd46a2046"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:41.020592 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:41.027737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:41.028222 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:41.028660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:41.029357 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:41.029808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:41.030264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:41.316488 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d9c65501-6d88-4b6a-90ba-597f463e0143 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.303202 Oct 13 04:27:41.317454 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 489/909] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:27:41 2022] POST /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments => generated 194 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:42.892445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-2133638351", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:42.945603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] validate_networks() for None {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:27:43.217907 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:27:43.218369 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:27:43.218842 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:27:43.219248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:27:43.219654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:27:43.228176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:43.228652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:43.229080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:43.229788 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:43.230258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:43.230676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:43.247402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Getting quotas for project cecec3eab7d3499a984c510355f61ebc. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:27:43.251184 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Getting quotas for user f38a674195a4438f87d467179d2c8860 and project cecec3eab7d3499a984c510355f61ebc. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:27:43.262290 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:27:43.264536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:27:43.309322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1850069b-5ec1-4e54-8056-e3f0003392de tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.420649 Oct 13 04:27:43.309980 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 421/910] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:27:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 422 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:27:43.322557 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-779704ba-9c1e-43b6-a7ed-7241c296926d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:43.323257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-779704ba-9c1e-43b6-a7ed-7241c296926d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:43.378095 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-779704ba-9c1e-43b6-a7ed-7241c296926d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:43.385492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-779704ba-9c1e-43b6-a7ed-7241c296926d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1215 microversion: 2.1 time: 0.068218 Oct 13 04:27:43.386062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 490/911] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:43 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1215 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:44.408656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:44.409378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:44.417268 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:44.417865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:44.418398 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:44.419163 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:44.419679 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:44.420163 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:44.482583 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:44.490052 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c834b47d-a0c9-4d6b-bacf-5453e14b1a2b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1267 microversion: 2.1 time: 0.085761 Oct 13 04:27:44.490647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 422/912] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:44 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1267 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:45.513189 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:45.513793 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:45.521572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:45.522059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:45.522505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:45.524044 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:45.524678 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:45.525105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:45.595899 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:45.606958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2f5ef47a-77f7-4255-8fb7-5466be2b371c tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1273 microversion: 2.1 time: 0.097553 Oct 13 04:27:45.607801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 491/913] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:45 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1273 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:46.197715 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:46.198853 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:46.205772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.206576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.207119 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.208026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.208594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.209058 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.307798 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'services' on Instance uuid 4f094e19-297d-4829-8af5-55be12ed8ee2 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:27:46.382362 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-539b0f07-b9aa-4fe1-a658-74308f9dcb6d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments/9aa52298-602c-481e-ae1b-4293f54da6da" status: 202 len: 0 microversion: 2.1 time: 0.188314 Oct 13 04:27:46.383376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 423/914] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:27:46 2022] DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments/9aa52298-602c-481e-ae1b-4293f54da6da => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 13 04:27:46.398232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:46.400539 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:46.414712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.415561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.416183 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.417142 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.418433 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.419161 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.459554 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4ba60e24-5d33-45a1-9c9d-024971e241b1 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071704 Oct 13 04:27:46.460273 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 492/915] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:46 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:46.631932 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:46.632659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:46.638900 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.639387 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.639846 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.640546 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.641001 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.641427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.843506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:46.851707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-24c0f271-3f80-4213-ab9b-ccc9702a702b tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.223449 Oct 13 04:27:46.852293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 424/916] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:46 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:46.941282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-1beff749-a712-455b-9c43-0a20d4123912 req-16cf7996-1fa7-4f0e-8fcc-859bdd17583d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "02706563-2d6b-46d7-84e0-595f6d284629", "tag": "21dc8ad2-d71a-425a-9b9d-dd93db542e7a"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:46.952048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-1beff749-a712-455b-9c43-0a20d4123912 req-16cf7996-1fa7-4f0e-8fcc-859bdd17583d service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:46.952804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-1beff749-a712-455b-9c43-0a20d4123912 req-16cf7996-1fa7-4f0e-8fcc-859bdd17583d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:46.953466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-1beff749-a712-455b-9c43-0a20d4123912 req-16cf7996-1fa7-4f0e-8fcc-859bdd17583d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:46.965580 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-1beff749-a712-455b-9c43-0a20d4123912 req-16cf7996-1fa7-4f0e-8fcc-859bdd17583d service nova] Creating event network-changed:21dc8ad2-d71a-425a-9b9d-dd93db542e7a for instance 02706563-2d6b-46d7-84e0-595f6d284629 on hv-cinder-86099 Oct 13 04:27:46.970436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-1beff749-a712-455b-9c43-0a20d4123912 req-16cf7996-1fa7-4f0e-8fcc-859bdd17583d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032905 Oct 13 04:27:46.971032 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 493/917] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:27:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:47.471730 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:47.474054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:47.481238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:47.481763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:47.482299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:47.483074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:47.483621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:47.484230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:47.515550 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-2c4cb334-1b55-4699-b47d-aea88c070118 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047919 Oct 13 04:27:47.516104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 425/918] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:47 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:47.874660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:47.875386 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:47.881495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:47.881976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:47.882463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:47.883146 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:47.883599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:47.884020 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:48.014586 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:48.022136 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-60158782-22c0-4f12-98ef-4815f35745b3 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.151791 Oct 13 04:27:48.023332 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 494/919] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:47 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:48.528252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:48.530474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:48.536586 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:48.537064 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:48.537501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:48.538178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:48.538879 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:48.539293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:48.571227 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d08ecc6d-510f-40c5-9045-f1cda2f98b30 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046822 Oct 13 04:27:48.571698 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 426/920] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:48 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:49.045158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:49.045765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:49.052157 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:49.052635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:49.053064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:49.053743 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:49.054773 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:49.055303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:49.183175 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:49.190754 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9f3871d5-2ee6-4183-82e0-5337f50ad3eb tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.149481 Oct 13 04:27:49.191321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 495/921] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:49 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:49.582629 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:49.584821 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:49.591960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:49.592434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:49.592857 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:49.593529 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:49.593974 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:49.594434 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:49.623107 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-320d6d8c-4ffa-44da-b960-5d8ffe5afd53 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044366 Oct 13 04:27:49.623651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 427/922] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:49 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:50.213059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:50.213674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:50.219790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:50.220281 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:50.220718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:50.221422 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:50.221874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:50.222324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:50.352958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:50.360819 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-35e7c93d-56f7-4e88-8778-411651652378 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.151845 Oct 13 04:27:50.361380 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 496/923] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:50 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:50.635864 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:50.638421 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:50.644454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:50.644933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:50.645358 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:50.646037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:50.646509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:50.646924 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:50.677695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6ba9a6b8-e177-44c2-98d1-e5d1795c2fab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045798 Oct 13 04:27:50.678184 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 428/924] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:50 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:27:51.383723 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:51.384526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:51.387016 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=38 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:51.392305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:51.392792 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:51.393230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:51.393948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:51.394450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:51.394873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:51.527186 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:51.534810 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a63ba7ec-5490-4d8b-b3dd-be684a682f27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.155662 Oct 13 04:27:51.535376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 497/925] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:51 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:51.690437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:51.692483 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:51.699104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:51.699595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:51.700068 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:51.700763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:51.701219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:51.701646 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:51.731228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-da330be5-ea70-4149-8049-11c9bce2c98e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044660 Oct 13 04:27:51.731787 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 429/926] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:51 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:27:52.230578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-c26c3c2e-5509-493e-a7a5-35cdbd6aa203 req-7be2ad3c-f10a-4553-8692-efd63f26dff3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02706563-2d6b-46d7-84e0-595f6d284629", "name": "network-vif-plugged", "status": "completed", "tag": "21dc8ad2-d71a-425a-9b9d-dd93db542e7a"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:52.240455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c26c3c2e-5509-493e-a7a5-35cdbd6aa203 req-7be2ad3c-f10a-4553-8692-efd63f26dff3 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:52.240936 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c26c3c2e-5509-493e-a7a5-35cdbd6aa203 req-7be2ad3c-f10a-4553-8692-efd63f26dff3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:52.241364 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c26c3c2e-5509-493e-a7a5-35cdbd6aa203 req-7be2ad3c-f10a-4553-8692-efd63f26dff3 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:52.253930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-c26c3c2e-5509-493e-a7a5-35cdbd6aa203 req-7be2ad3c-f10a-4553-8692-efd63f26dff3 service nova] Creating event network-vif-plugged:21dc8ad2-d71a-425a-9b9d-dd93db542e7a for instance 02706563-2d6b-46d7-84e0-595f6d284629 on hv-cinder-86099 Oct 13 04:27:52.259948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-c26c3c2e-5509-493e-a7a5-35cdbd6aa203 req-7be2ad3c-f10a-4553-8692-efd63f26dff3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033966 Oct 13 04:27:52.260741 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 498/927] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:27:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:52.559074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:52.559678 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:52.565891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:52.566531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:52.567746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:52.568958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:52.568958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:52.569299 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:52.574531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:52.745305 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:52.747602 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:52.753652 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:52.754143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:52.754627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:52.755328 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:52.755748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:52.756173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:52.756173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:52.764069 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c4be9f85-9e07-4d20-9791-6a9c0a8080ce tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.209668 Oct 13 04:27:52.764669 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 430/928] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:52 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:52.789927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ea724427-2377-4467-a4ee-c3ce7d224938 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048958 Oct 13 04:27:52.790549 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 499/929] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:52 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:27:53.127202 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:53.129341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:53.135303 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.135791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.136232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.136922 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.137379 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.137806 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.167418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-465493ce-dbba-4449-8cce-830fb2ebedf4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044121 Oct 13 04:27:53.167978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 431/930] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:53 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:53.178977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:53.179462 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:53.185897 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.186415 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.186849 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.187536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.187989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.188402 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.218250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4c5fd84f-f463-4fae-a636-443ddef9a3c1 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/4dabbe27-59a3-4a93-a5a0-7c1dd46a2046" status: 200 len: 194 microversion: 2.1 time: 0.043504 Oct 13 04:27:53.218843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 500/931] 10.222.0.27 () {58 vars in 1413 bytes} [Thu Oct 13 04:27:53 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/4dabbe27-59a3-4a93-a5a0-7c1dd46a2046 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:53.788219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:53.788837 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:53.791267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:53.791926 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:53.798634 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.799123 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.799556 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.800251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.800705 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.801128 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.802864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:53.805802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:53.807995 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:53.814361 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.814938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.815337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.816079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.816531 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.816956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.832630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:53.851215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-455ddebd-f9ba-47cc-8d90-bbe2319d925d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052819 Oct 13 04:27:53.852644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 501/932] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:27:53 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:27:53.919523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:53.920023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:53.927040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.927540 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.928031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.928846 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:53.929172 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:53.929814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:53.951344 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:53.956552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:27:53.958909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-27595d82-0f69-4543-a819-a7921210d8f1 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1315 microversion: 2.1 time: 0.174610 Oct 13 04:27:53.959488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 432/933] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:53 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1315 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:54.031442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-de4f8609-0ea4-437f-9597-6cdbb5a5d1ed tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 204 len: 0 microversion: 2.1 time: 0.116589 Oct 13 04:27:54.032011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 502/934] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:27:53 2022] DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 13 04:27:54.040858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:54.041472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:54.047289 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:54.047775 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:54.048211 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:54.048896 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:54.049354 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:54.049780 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:54.180920 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:54.188591 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4efca283-8d41-4b4b-9162-7c4128a24a5e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1683 microversion: 2.1 time: 0.152290 Oct 13 04:27:54.189158 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 433/935] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:54 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1683 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:54.557063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "65596bd0-92f0-481c-90d9-fa1b718d0050"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:54.559471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:54.565331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:54.565815 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:54.566286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:54.566975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:54.567428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:54.567852 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:54.982726 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:54.983262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:54.989540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:54.990018 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:54.990475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:54.991155 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:54.991601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:54.992021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.122233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:55.130851 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f9254d1b-ff49-42c2-970c-85d8f02c9329 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1465 microversion: 2.1 time: 0.152316 Oct 13 04:27:55.131440 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 434/936] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:54 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1465 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:27:55.176468 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:55.176959 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:55.183296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:55.183769 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:55.184197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.184873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:55.185320 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:55.185736 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.212038 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:27:55.440062 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-48e088cc-dba2-4baa-9875-459f559a4810 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "DELETE /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 204 len: 0 microversion: 2.1 time: 0.267137 Oct 13 04:27:55.440640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 435/937] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:27:55 2022] DELETE /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 13 04:27:55.447447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:55.448174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:55.454307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:55.454797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:55.455233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.455921 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:55.456372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:55.456795 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.595322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:55.603596 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9f81fd20-c2b2-43c0-aa1c-9bd5d5be16a6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1683 microversion: 2.1 time: 0.160242 Oct 13 04:27:55.604145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 436/938] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:55 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1683 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:55.608755 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:55.609349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:55.614860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:55.615338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:55.615771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.616450 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:55.616897 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:55.617318 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:55.749750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:55.757357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-01bea463-1053-479e-aab8-363ce120168d tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1471 microversion: 2.1 time: 0.152295 Oct 13 04:27:55.757927 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 437/939] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:55 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1471 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:56.628116 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:56.628764 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:56.636618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:56.637092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:56.637518 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:56.638256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:56.638708 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:56.639152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:56.770617 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:56.778420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5d2b5d90-7836-4ee8-b7c8-5b3108cd8c28 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1683 microversion: 2.1 time: 0.154158 Oct 13 04:27:56.778971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 438/940] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:56 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1683 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:27:56.783273 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:56.783868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:56.789832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:56.790539 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:56.790968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:56.791650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:56.792098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:56.792519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:56.920271 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:56.927852 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c6fa6487-eb83-4bbe-91cb-71e16cccde27 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1471 microversion: 2.1 time: 0.147959 Oct 13 04:27:56.928410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 439/941] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:56 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1471 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:57.432005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e2b17f16-392c-4d8f-a26a-839e092df00e tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "POST /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.879164 Oct 13 04:27:57.432701 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 503/942] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:27:54 2022] POST /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments => generated 194 bytes in 2881 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:57.801983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:57.802636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:57.805960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:27:57.811338 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:57.811895 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:57.812335 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:57.813037 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:57.813492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:57.813978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:57.952220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:57.952914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:57.960021 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:57.961599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:57.962118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:57.963677 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:57.964731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:57.964731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:58.067492 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:58.075719 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-06323094-e08a-40ec-bb26-ce1d5c4087ba tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1683 microversion: 2.1 time: 0.277583 Oct 13 04:27:58.076662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 440/943] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:57 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1683 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:27:58.108140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f38a674195a4438f87d467179d2c8860', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cecec3eab7d3499a984c510355f61ebc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:58.116694 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9037ed61-d899-4c1a-8ab1-4e393896df94 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 200 len: 1471 microversion: 2.1 time: 0.169246 Oct 13 04:27:58.117788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 504/944] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:57 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 1471 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:27:58.241042 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-ef4c96ad-3350-4c20-b158-867f51dcab9a req-b008e8b9-915d-4f6c-a65a-521c6763f458 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02706563-2d6b-46d7-84e0-595f6d284629", "name": "network-vif-deleted", "tag": "21dc8ad2-d71a-425a-9b9d-dd93db542e7a"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:27:58.256002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ef4c96ad-3350-4c20-b158-867f51dcab9a req-b008e8b9-915d-4f6c-a65a-521c6763f458 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:58.260074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ef4c96ad-3350-4c20-b158-867f51dcab9a req-b008e8b9-915d-4f6c-a65a-521c6763f458 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:58.260659 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-ef4c96ad-3350-4c20-b158-867f51dcab9a req-b008e8b9-915d-4f6c-a65a-521c6763f458 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:58.291026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-ef4c96ad-3350-4c20-b158-867f51dcab9a req-b008e8b9-915d-4f6c-a65a-521c6763f458 service nova] Creating event network-vif-deleted:21dc8ad2-d71a-425a-9b9d-dd93db542e7a for instance 02706563-2d6b-46d7-84e0-595f6d284629 on hv-cinder-86099 Oct 13 04:27:58.295929 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-ef4c96ad-3350-4c20-b158-867f51dcab9a req-b008e8b9-915d-4f6c-a65a-521c6763f458 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058962 Oct 13 04:27:58.296525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 441/945] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:27:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:27:59.105028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:59.105645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:59.112494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:59.112984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:59.113536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:59.114260 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:59.114905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:59.115365 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:59.143445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:27:59.145182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] [instance: 02706563-2d6b-46d7-84e0-595f6d284629] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:27:59.154508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:59.154798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:59.155225 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:59.155363 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:59.155715 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:59.156051 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:59.156173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lazy-loading 'fault' on Instance uuid 4f094e19-297d-4829-8af5-55be12ed8ee2 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:27:59.156860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:27:59.157380 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:27:59.157868 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:27:59.172148 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] HTTP exception thrown: Instance 02706563-2d6b-46d7-84e0-595f6d284629 could not be found. Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Request method failure captured: Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 HTTP/1.1 Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5O9_nDDiTlMHCnTMaNwkSS410KinBmedTMaeCNqlo6wWr49D-ThPiB3a6VlkdTcasexNuT46iOHvYx8DlWMVqlDJgTYYquf2AGtnlOBfFJJMkFXMUqFiLaL4zppWoYhXJ7Ph8ofizKPVES_q900j1Q5DJlOFBwSUzDyIW9TdLG1O1U Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: cecec3eab7d3499a984c510355f61ebc Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesNegativeTest-762276871 Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:27:59.174006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/cecec3eab7d3499a984c510355f61ebc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/cecec3eab7d3499a984c510355f61ebc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/cecec3eab7d3499a984c510355f61ebc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_cecec3eab7d3499a984c510355f61ebc", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesNegativeTest-762276871 Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: cecec3eab7d3499a984c510355f61ebc Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesNegativeTest-762276871 Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesNegativeTest-762276871-project Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: f38a674195a4438f87d467179d2c8860 Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesNegativeTest-762276871-project Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 02706563-2d6b-46d7-84e0-595f6d284629 could not be found. Oct 13 04:27:59.175874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '02706563-2d6b-46d7-84e0-595f6d284629'} Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 02706563-2d6b-46d7-84e0-595f6d284629 could not be found. Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:27:59.177488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 02706563-2d6b-46d7-84e0-595f6d284629 could not be found. Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 02706563-2d6b-46d7-84e0-595f6d284629 could not be found. Oct 13 04:27:59.179030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:27:59.181562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] Returning 404 to user: Instance 02706563-2d6b-46d7-84e0-595f6d284629 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:27:59.182558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aa8ab205-90ee-41a6-b4d3-ebd05a93ef03 tempest-VolumesNegativeTest-762276871 tempest-VolumesNegativeTest-762276871-project] 10.222.0.27 "GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629" status: 404 len: 111 microversion: 2.1 time: 0.044119 Oct 13 04:27:59.183196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 442/946] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:59 2022] GET /compute/v2.1/servers/02706563-2d6b-46d7-84e0-595f6d284629 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:27:59.202872 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999c89e1bc7a426886fc802e28d38596', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7397177414dd4ed5baed133cc89e7299', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:27:59.210862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-27151bef-bad9-4fad-9268-ba795b397e41 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 200 len: 1332 microversion: 2.1 time: 0.110035 Oct 13 04:27:59.214887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 505/947] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:27:59 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 1332 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:00.233602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:00.234233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:00.240524 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:00.241008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:00.241438 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:00.242768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:00.243389 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:00.243853 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 HTTP/1.1 Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:00.262750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:00.264382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '4f094e19-297d-4829-8af5-55be12ed8ee2'} Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:00.265306 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:00.266774 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:00.268028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:00.268028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-691d7cc1-5269-4bed-84ce-52ab09776629 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 404 len: 111 microversion: 2.1 time: 0.032975 Oct 13 04:28:00.268028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 443/948] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:00 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:00.275474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-18df41e5-3711-402b-8dc1-907213b9d6cf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '.version_select at 0x7f0794752d30>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:00.294342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-18df41e5-3711-402b-8dc1-907213b9d6cf tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1217271909" status: 202 len: 0 microversion: 2.1 time: 0.023184 Oct 13 04:28:00.294908 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 506/949] 10.222.0.27 () {58 vars in 1221 bytes} [Thu Oct 13 04:28:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1217271909 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:28:00.311431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-cdd18797-ebd1-4d24-826b-21cffe355547 req-515096ea-ff29-4c9e-8119-c6e92d1fd329 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f094e19-297d-4829-8af5-55be12ed8ee2", "name": "network-vif-deleted", "tag": "b7fcfa46-fbb2-466a-962d-bdf70352c79c"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:28:00.322149 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-cdd18797-ebd1-4d24-826b-21cffe355547 req-515096ea-ff29-4c9e-8119-c6e92d1fd329 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:00.322675 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-cdd18797-ebd1-4d24-826b-21cffe355547 req-515096ea-ff29-4c9e-8119-c6e92d1fd329 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:00.323107 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-cdd18797-ebd1-4d24-826b-21cffe355547 req-515096ea-ff29-4c9e-8119-c6e92d1fd329 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:00.335530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-cdd18797-ebd1-4d24-826b-21cffe355547 req-515096ea-ff29-4c9e-8119-c6e92d1fd329 service nova] Creating event network-vif-deleted:b7fcfa46-fbb2-466a-962d-bdf70352c79c for instance 4f094e19-297d-4829-8af5-55be12ed8ee2 on hv-cinder-86099 Oct 13 04:28:00.341266 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-cdd18797-ebd1-4d24-826b-21cffe355547 req-515096ea-ff29-4c9e-8119-c6e92d1fd329 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033461 Oct 13 04:28:00.345138 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 444/950] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:28:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:02.099539 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:02.100232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:02.110999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:02.110999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:02.110999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:02.111956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:02.112866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:02.113342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:02.131515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 HTTP/1.1 Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:02.134152 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.136071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '4f094e19-297d-4829-8af5-55be12ed8ee2'} Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:02.137340 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.138974 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.140572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:02.140572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ea97a24a-3462-4634-aa5c-8d292375da8e tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 404 len: 111 microversion: 2.1 time: 0.041719 Oct 13 04:28:02.142235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 507/951] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:02 2022] DELETE /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:02.148256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:02.149339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 4f094e19-297d-4829-8af5-55be12ed8ee2] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:02.155779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:02.156349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:02.156791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:02.157499 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:02.158013 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:02.158478 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:02.173469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 HTTP/1.1 Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:02.175209 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.176772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '4f094e19-297d-4829-8af5-55be12ed8ee2'} Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:02.177781 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. Oct 13 04:28:02.179298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:02.180792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 4f094e19-297d-4829-8af5-55be12ed8ee2 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:02.180792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-20cf5b2b-dd29-42d0-97bf-965ccf06c0ab tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2" status: 404 len: 111 microversion: 2.1 time: 0.033684 Oct 13 04:28:02.180792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 445/952] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:02 2022] GET /compute/v2.1/servers/4f094e19-297d-4829-8af5-55be12ed8ee2 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:28:03.343804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:03.344278 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:03.351061 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.352275 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.352712 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.353397 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.353845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.354298 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.367749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 HTTP/1.1 Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:03.369292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.370966 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '83b343e2-9707-46ae-a665-5aa2644cb682'} Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:03.371878 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.373358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.375669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:03.375669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b329c4d7-8f45-452d-a0e7-99e140c4df52 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 404 len: 111 microversion: 2.1 time: 0.031758 Oct 13 04:28:03.375669 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 508/953] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:03 2022] DELETE /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:03.379336 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:03.379969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 83b343e2-9707-46ae-a665-5aa2644cb682] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:03.386451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.386782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.387158 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.387840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.388287 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.388703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.409243 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 HTTP/1.1 Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:03.411228 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.412751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '83b343e2-9707-46ae-a665-5aa2644cb682'} Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:03.413619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. Oct 13 04:28:03.415308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.418035 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 83b343e2-9707-46ae-a665-5aa2644cb682 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:03.419520 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c87d630b-e47c-42ed-9ffc-5b1a8c055b0d tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682" status: 404 len: 111 microversion: 2.1 time: 0.044074 Oct 13 04:28:03.420230 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 446/954] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:03 2022] GET /compute/v2.1/servers/83b343e2-9707-46ae-a665-5aa2644cb682 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:28:03.429147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:03.430221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:03.436474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.436982 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.437405 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.438104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.438599 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.439039 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.453634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c HTTP/1.1 Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:03.455220 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.456788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'f3720a7c-f5a9-468e-8195-221790e8931c'} Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:03.457682 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.459159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.460583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:03.460583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c4ef64a3-fa31-42af-907d-404abc8cb4c9 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 404 len: 111 microversion: 2.1 time: 0.031693 Oct 13 04:28:03.460583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 509/955] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:03 2022] DELETE /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:28:03.469257 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:03.470076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: f3720a7c-f5a9-468e-8195-221790e8931c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:03.475989 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.476564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.477005 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.477697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:03.478142 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:03.478683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:03.491884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c HTTP/1.1 Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:03.494232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.495811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'f3720a7c-f5a9-468e-8195-221790e8931c'} Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:03.496647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. Oct 13 04:28:03.498092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:03.500087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance f3720a7c-f5a9-468e-8195-221790e8931c could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:03.500087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-10e194f8-2439-4752-8730-b9d2957debe6 tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c" status: 404 len: 111 microversion: 2.1 time: 0.030220 Oct 13 04:28:03.500087 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 447/956] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:03 2022] GET /compute/v2.1/servers/f3720a7c-f5a9-468e-8195-221790e8931c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:04.658073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:04.658591 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:04.665286 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:04.665764 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:04.666246 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:04.666927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:04.667379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:04.667801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:04.681634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 HTTP/1.1 Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:04.683191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.684729 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '62033c3e-e894-4560-95b8-34e25cb33e72'} Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:04.685614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.687128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.688476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:04.688476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5acf9923-9f3a-4fdd-9cc5-3e4234a4c0de tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 404 len: 111 microversion: 2.1 time: 0.030429 Oct 13 04:28:04.688476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 510/957] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:04 2022] DELETE /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:04.692945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:04.693558 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] [instance: 62033c3e-e894-4560-95b8-34e25cb33e72] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:04.700835 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:04.701316 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:04.701749 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:04.702466 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:04.702943 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:04.703372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:04.716686 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] HTTP exception thrown: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Request method failure captured: Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 HTTP/1.1 Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5IqosSFawPJ8PKJiHFEqrUQWDCeoBT35m67DRj8EC99dOTbCs8Bel1RchycrYAM-DA2Ilr598bOr1pCDdjgVCyKrz2Eqfi5Kk8SamfMh4vUFQxKkpZmO161MtlXdhGNP0FPm2-e-HTC4VjzKnihvpFJpdl2jQ8vkZ3H0Fpa5KwsHCM Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:04.718193 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7397177414dd4ed5baed133cc89e7299"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7397177414dd4ed5baed133cc89e7299", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7397177414dd4ed5baed133cc89e7299 Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-864052640 Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 999c89e1bc7a426886fc802e28d38596 Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeNegativeTest-864052640-project Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.719743 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '62033c3e-e894-4560-95b8-34e25cb33e72'} Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:04.720564 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. Oct 13 04:28:04.721784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:04.722969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] Returning 404 to user: Instance 62033c3e-e894-4560-95b8-34e25cb33e72 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:04.722969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a737e1c6-b8d8-49db-806e-b474f5f8dc2c tempest-AttachVolumeNegativeTest-864052640 tempest-AttachVolumeNegativeTest-864052640-project] 10.222.0.27 "GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72" status: 404 len: 111 microversion: 2.1 time: 0.030509 Oct 13 04:28:04.722969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 448/958] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:04 2022] GET /compute/v2.1/servers/62033c3e-e894-4560-95b8-34e25cb33e72 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:05.036381 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:05.038667 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:05.044970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:05.045461 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:05.046108 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:05.046842 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:05.047301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:05.047732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:05.079235 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-356e2359-d72c-441c-b33a-d08f95942bbf tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.046849 Oct 13 04:28:05.080406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 511/959] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:28:05 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments => generated 371 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:05.221594 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:05.222091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:05.228130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:05.228621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:05.229061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:05.229759 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:05.230242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:05.230695 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:05.260046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7ab64fbf-2eae-4d9e-a29c-f8c2e66ef14f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/4dabbe27-59a3-4a93-a5a0-7c1dd46a2046" status: 200 len: 194 microversion: 2.1 time: 0.042436 Oct 13 04:28:05.260633 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 449/960] 10.222.0.27 () {58 vars in 1413 bytes} [Thu Oct 13 04:28:05 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/4dabbe27-59a3-4a93-a5a0-7c1dd46a2046 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:05.271241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:05.271743 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:05.278803 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:05.279287 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:05.279823 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:05.280572 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:05.281112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:05.281647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:05.364151 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lazy-loading 'services' on Instance uuid 17727b50-e3a3-48f2-a277-7e2f2ff16903 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:28:05.451108 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4f4dc273-22aa-4c45-8139-ba373421cacc tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/4dabbe27-59a3-4a93-a5a0-7c1dd46a2046" status: 202 len: 0 microversion: 2.1 time: 0.183609 Oct 13 04:28:05.451691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 512/961] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:28:05 2022] DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/4dabbe27-59a3-4a93-a5a0-7c1dd46a2046 => generated 0 bytes in 185 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:28:07.681593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:07.682174 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:07.689709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:07.690725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:07.691457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:07.692446 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:07.693307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:07.694025 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:07.716285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:28:07.739823 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9712a206-9d0d-4a4b-9d85-f409406dad04 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/65596bd0-92f0-481c-90d9-fa1b718d0050" status: 200 len: 194 microversion: 2.1 time: 0.063770 Oct 13 04:28:07.741101 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 450/962] 10.222.0.27 () {58 vars in 1413 bytes} [Thu Oct 13 04:28:07 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/65596bd0-92f0-481c-90d9-fa1b718d0050 => generated 194 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:07.753495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:07.754770 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:07.767181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:07.767922 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:07.768541 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:07.769428 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:07.769881 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:07.770509 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:07.876778 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lazy-loading 'services' on Instance uuid 17727b50-e3a3-48f2-a277-7e2f2ff16903 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:28:07.973233 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-03aed01f-d8a4-41fc-87dc-62c8f2ef239c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/65596bd0-92f0-481c-90d9-fa1b718d0050" status: 202 len: 0 microversion: 2.1 time: 0.224898 Oct 13 04:28:07.974379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 513/963] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:28:07 2022] DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments/65596bd0-92f0-481c-90d9-fa1b718d0050 => generated 0 bytes in 227 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:28:09.166791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:09.168850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:09.175161 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.175656 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.176093 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.176791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.177272 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.177693 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.206826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-66f281ee-ec38-4766-8aed-0d4ffe96561c tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044396 Oct 13 04:28:09.207300 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 451/964] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:28:09 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:28:09.327459 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:09.329809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:09.336499 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.337252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.337411 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.338182 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.338658 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.339080 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.371890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-365726cc-e10f-4943-adaf-3c0e12cc39c3 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048616 Oct 13 04:28:09.372627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 514/965] 10.222.0.27 () {58 vars in 1302 bytes} [Thu Oct 13 04:28:09 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 13 04:28:09.437394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:09.437893 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:09.443822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.444310 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.444751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.445568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.446026 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.446477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.472603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:28:09.552068 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9d6a9f1c-2a94-46c1-8692-901a78741a23 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 204 len: 0 microversion: 2.1 time: 0.118160 Oct 13 04:28:09.553172 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 452/966] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:09 2022] DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:28:09.562409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:09.563015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:09.569017 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.569519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.569950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.571448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:09.572062 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:09.572519 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:09.712434 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:09.720239 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-8f3f4b16-c1ab-4475-9dc9-d5baefc52f6f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1673 microversion: 2.1 time: 0.162090 Oct 13 04:28:09.720804 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 515/967] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:09 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1673 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:10.745775 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:10.746502 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:10.752779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:10.753259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:10.753685 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:10.754397 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:10.754845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:10.755253 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:10.865750 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:10.868516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:10.875071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:10.875603 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:10.876301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:10.877204 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:10.877894 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:10.878490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:10.892443 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:10.901923 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-5ccb9699-cd9b-4c30-aae3-8fdbf2d35bfb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1673 microversion: 2.1 time: 0.160958 Oct 13 04:28:10.902528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 453/968] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:10 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1673 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:10.914813 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-81d47e58-eb5c-4aa4-9a3e-6729b471bbec tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.053162 Oct 13 04:28:10.915383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 516/969] 10.222.0.27 () {60 vars in 1343 bytes} [Thu Oct 13 04:28:10 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:28:10.995769 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:10.996180 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:11.002111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:11.002626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:11.003061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:11.003751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:11.004200 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:11.004619 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:11.031242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:28:11.109874 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-326f35d2-fec0-4eb7-90a5-8aa367ac9d9e tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 204 len: 0 microversion: 2.20 time: 0.119840 Oct 13 04:28:11.113621 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 454/970] 10.222.0.27 () {60 vars in 1280 bytes} [Thu Oct 13 04:28:10 2022] DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 13 04:28:11.120182 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:11.120983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:11.130850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:11.131491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:11.132089 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:11.133858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:11.134463 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:11.134984 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:11.139718 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=3,nova_cell1:INSERT=4 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:28:11.278843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:11.287327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:11.289376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:11.290641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4f5d151f-0b68-4c8b-864f-2775c9709132 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.175680 Oct 13 04:28:11.291228 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 517/971] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:11 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:28:11.927548 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:11.928236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:11.934867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:11.935399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:11.935766 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:11.936455 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:11.936979 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:11.937436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:12.075603 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:12.084971 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-aba07cdc-8543-4ea9-95fd-94e13b862bda tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1673 microversion: 2.1 time: 0.161464 Oct 13 04:28:12.085608 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 455/972] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:11 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1673 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:12.317418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:12.318113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:12.324784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:12.325277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:12.325717 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:12.326446 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:12.326902 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:12.327337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:12.466425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:12.475634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:12.477185 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:12.478425 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6b6aa884-34f0-4230-9390-df97f9bee4c2 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.165242 Oct 13 04:28:12.478950 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 518/973] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:12 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:28:13.111153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:13.112351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:13.121739 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:13.135158 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:13.136853 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:13.139522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:13.140004 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:13.140457 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:13.437954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:13.446880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-735058b9-12aa-456b-bbd0-8a15311ca93b tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1673 microversion: 2.1 time: 0.341170 Oct 13 04:28:13.447986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 456/974] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:13 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1673 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:28:13.505618 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:13.506231 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:13.513358 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:13.513850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:13.514318 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:13.515025 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:13.515516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:13.515945 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:13.653791 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:13.662067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:13.663635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:13.665000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-85094804-793d-4c89-8f39-f2f6da6416e5 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1724 microversion: 2.20 time: 0.163351 Oct 13 04:28:13.665643 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 519/975] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:13 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:28:14.472100 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:14.472717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:14.501581 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:14.503157 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:14.503846 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:14.506601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:14.509274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:14.510014 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:14.697158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:14.700014 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:14.716216 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:14.716999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:14.718139 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:14.719087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:14.719733 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:14.720383 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:14.738612 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:14.751408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f894ffb5-c40a-46de-82dc-659c8346bfef tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1606 microversion: 2.1 time: 0.283072 Oct 13 04:28:14.764219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 457/976] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:14 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1606 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:14.774003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-77fe1d81-2f6d-4dfa-9bf6-dcb73d21536c req-451800e0-61da-443c-91eb-66c8a00cf9cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "499d8484-0292-40e2-a084-f94dac55a15d", "name": "network-vif-deleted", "tag": "fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:28:14.786244 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-77fe1d81-2f6d-4dfa-9bf6-dcb73d21536c req-451800e0-61da-443c-91eb-66c8a00cf9cd service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:14.787165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-77fe1d81-2f6d-4dfa-9bf6-dcb73d21536c req-451800e0-61da-443c-91eb-66c8a00cf9cd service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:14.787754 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-77fe1d81-2f6d-4dfa-9bf6-dcb73d21536c req-451800e0-61da-443c-91eb-66c8a00cf9cd service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:14.814472 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-77fe1d81-2f6d-4dfa-9bf6-dcb73d21536c req-451800e0-61da-443c-91eb-66c8a00cf9cd service nova] Creating event network-vif-deleted:fee760d7-9ab4-4c57-b0d2-4de08aa5cfa5 for instance 499d8484-0292-40e2-a084-f94dac55a15d on hv-cinder-86099 Oct 13 04:28:14.821195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-77fe1d81-2f6d-4dfa-9bf6-dcb73d21536c req-451800e0-61da-443c-91eb-66c8a00cf9cd service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052410 Oct 13 04:28:14.822339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 458/977] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:28:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:14.849075 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:14.861835 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:14.863784 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '35a058c849104a9699ecfcd0c81d398e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75387c76ea384e5c96a336751c613e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:14.865288 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-110ea622-a16c-48ca-8126-29d7ca7bfe44 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 200 len: 1656 microversion: 2.20 time: 0.177690 Oct 13 04:28:14.866490 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 520/978] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:14 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 1656 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 13 04:28:15.777176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:15.777983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:15.785265 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:15.786427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:15.787252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:15.788282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:15.789703 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:15.790724 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:15.854275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:15.859281 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:15.860099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:15.862177 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lazy-loading 'fault' on Instance uuid 17727b50-e3a3-48f2-a277-7e2f2ff16903 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:28:15.893048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:15.893725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:15.904758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:15.905249 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:15.905681 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:15.906421 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:15.909809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:15.910324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:15.928668 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f6cf725908041ccaceb1ffa694ad161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d6c8ffc0354e9d889b1abc471602ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:15.935006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] HTTP exception thrown: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Request method failure captured: Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d HTTP/1.1 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I1cSk8hX3JU_WckuMYN6y60NIhVtGTpF96ZJ00QD5DtLllxQHaJ8R-i05HsPJElOW6QbgdwYhmTtXPrdAxugZ1ndGxeGo8Zr5D2Ef2fcmlG0zmhRrMJz9O1gXDAcU7KNfBOv9tS97zox4jQeP-5hUoCp4lCJTvUnwozSUO8L5hUF4 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Openstack-Nova-Api-Version: 2.20 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:15.937256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_75387c76ea384e5c96a336751c613e6e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 35a058c849104a9699ecfcd0c81d398e Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:15.938672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '499d8484-0292-40e2-a084-f94dac55a15d'} Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:15.939821 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:15.941002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:15.942066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-10c91573-c9d8-4b0f-94e6-ec907ac38710 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 200 len: 1327 microversion: 2.1 time: 0.165759 Oct 13 04:28:15.942066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 459/979] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:15 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 1327 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:15.944119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Returning 404 to user: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:15.947099 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-21292f3b-fa7c-41ea-ac40-c9cd9607bb54 tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 404 len: 111 microversion: 2.20 time: 0.057158 Oct 13 04:28:15.948408 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 521/980] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:15 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 13 04:28:15.957740 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cf7d460b-e0ba-489c-a396-da5be00475dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '.version_select at 0x7f0794a21c10>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:15.982305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cf7d460b-e0ba-489c-a396-da5be00475dc tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-128532449" status: 204 len: 0 microversion: 2.20 time: 0.028291 Oct 13 04:28:15.983003 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 460/981] 10.222.0.27 () {60 vars in 1259 bytes} [Thu Oct 13 04:28:15 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-128532449 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 13 04:28:16.838136 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-89b7b8db-b1f1-4f7e-80e3-19173260b93e req-b2c01ab8-1eb1-4c52-91b2-486acb395661 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "17727b50-e3a3-48f2-a277-7e2f2ff16903", "name": "network-vif-deleted", "tag": "df61582b-5a9b-4646-8d63-4fa70aa7b3ab"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:28:16.847921 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-89b7b8db-b1f1-4f7e-80e3-19173260b93e req-b2c01ab8-1eb1-4c52-91b2-486acb395661 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:16.848375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-89b7b8db-b1f1-4f7e-80e3-19173260b93e req-b2c01ab8-1eb1-4c52-91b2-486acb395661 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:16.848911 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-89b7b8db-b1f1-4f7e-80e3-19173260b93e req-b2c01ab8-1eb1-4c52-91b2-486acb395661 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:16.860301 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-89b7b8db-b1f1-4f7e-80e3-19173260b93e req-b2c01ab8-1eb1-4c52-91b2-486acb395661 service nova] Creating event network-vif-deleted:df61582b-5a9b-4646-8d63-4fa70aa7b3ab for instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 on hv-cinder-86099 Oct 13 04:28:16.865225 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-89b7b8db-b1f1-4f7e-80e3-19173260b93e req-b2c01ab8-1eb1-4c52-91b2-486acb395661 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031013 Oct 13 04:28:16.865811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 522/982] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:28:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:16.964505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:16.965124 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:16.972017 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:16.972513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:16.972952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:16.973684 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:16.974141 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:16.974596 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:16.988618 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] HTTP exception thrown: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Request method failure captured: Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 HTTP/1.1 Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I2HLU7FfvJKR98uwPjziNoRi-plQYU_qa31AUawJJfQTDe0ClpEi_AolcXzaGTrMWjpZg5jnnUmsfHEhTDA64zZTOxf6vR_6v7pZrQ5a3ydCzLvSN-EpnkKygAetOOpdbebnBRjk4j66fWOy2g5-ZRtt7NnJKkv9ExzynEwDipG9o Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:16.990314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_73d6c8ffc0354e9d889b1abc471602ac", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 1f6cf725908041ccaceb1ffa694ad161 Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:16.992372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '17727b50-e3a3-48f2-a277-7e2f2ff16903'} Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:16.993267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:16.995029 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:16.996422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Returning 404 to user: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:16.996422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-38c0a5d1-7b3a-46f8-8cbb-162774a326e9 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 404 len: 111 microversion: 2.1 time: 0.033140 Oct 13 04:28:16.996422 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 461/983] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:16 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 13 04:28:17.003946 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-dc62ea09-6b8d-4391-8182-6f1cc66c1d3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '.version_select at 0x7f0794b0d280>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:17.023753 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-dc62ea09-6b8d-4391-8182-6f1cc66c1d3f tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1578838222" status: 202 len: 0 microversion: 2.1 time: 0.025024 Oct 13 04:28:17.024637 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 523/984] 10.222.0.27 () {58 vars in 1221 bytes} [Thu Oct 13 04:28:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1578838222 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:28:17.789802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:17.789802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:17.795954 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:17.796573 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:17.797197 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:17.798506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:17.798970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:17.799405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:17.813934 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] HTTP exception thrown: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Request method failure captured: Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d HTTP/1.1 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I1cSk8hX3JU_WckuMYN6y60NIhVtGTpF96ZJ00QD5DtLllxQHaJ8R-i05HsPJElOW6QbgdwYhmTtXPrdAxugZ1ndGxeGo8Zr5D2Ef2fcmlG0zmhRrMJz9O1gXDAcU7KNfBOv9tS97zox4jQeP-5hUoCp4lCJTvUnwozSUO8L5hUF4 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Openstack-Nova-Api-Version: 2.20 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:17.815653 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_75387c76ea384e5c96a336751c613e6e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 35a058c849104a9699ecfcd0c81d398e Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.817260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '499d8484-0292-40e2-a084-f94dac55a15d'} Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:17.818152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.819510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.821477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Returning 404 to user: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:17.821477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-4aa1129c-2c14-42e5-958b-d80faa1cc06f tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 404 len: 111 microversion: 2.20 time: 0.033774 Oct 13 04:28:17.821477 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 462/985] 10.222.0.27 () {60 vars in 1280 bytes} [Thu Oct 13 04:28:17 2022] DELETE /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:28:17.827483 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:17.828209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: 499d8484-0292-40e2-a084-f94dac55a15d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:17.835003 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:17.835553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:17.836187 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:17.836973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:17.837430 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:17.837934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:17.854092 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] HTTP exception thrown: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Request method failure captured: Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d HTTP/1.1 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I1cSk8hX3JU_WckuMYN6y60NIhVtGTpF96ZJ00QD5DtLllxQHaJ8R-i05HsPJElOW6QbgdwYhmTtXPrdAxugZ1ndGxeGo8Zr5D2Ef2fcmlG0zmhRrMJz9O1gXDAcU7KNfBOv9tS97zox4jQeP-5hUoCp4lCJTvUnwozSUO8L5hUF4 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Openstack-Nova-Api-Version: 2.20 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:17.855774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_75387c76ea384e5c96a336751c613e6e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 35a058c849104a9699ecfcd0c81d398e Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.857876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '499d8484-0292-40e2-a084-f94dac55a15d'} Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:17.858874 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. Oct 13 04:28:17.860252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:17.861633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Returning 404 to user: Instance 499d8484-0292-40e2-a084-f94dac55a15d could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:17.861633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-78c88e06-0302-4a3f-9d2c-1ed84a39708d tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d" status: 404 len: 111 microversion: 2.20 time: 0.033854 Oct 13 04:28:17.861633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 524/986] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:17 2022] GET /compute/v2.1/servers/499d8484-0292-40e2-a084-f94dac55a15d => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:28:19.000111 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:19.000607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:19.007076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:19.007605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:19.008047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:19.008746 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:19.009214 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:19.009641 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:19.024680 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] HTTP exception thrown: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Request method failure captured: Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 HTTP/1.1 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I1cSk8hX3JU_WckuMYN6y60NIhVtGTpF96ZJ00QD5DtLllxQHaJ8R-i05HsPJElOW6QbgdwYhmTtXPrdAxugZ1ndGxeGo8Zr5D2Ef2fcmlG0zmhRrMJz9O1gXDAcU7KNfBOv9tS97zox4jQeP-5hUoCp4lCJTvUnwozSUO8L5hUF4 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Openstack-Nova-Api-Version: 2.20 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:19.026264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_75387c76ea384e5c96a336751c613e6e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 35a058c849104a9699ecfcd0c81d398e Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.027967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'cc378324-badd-4a63-ba04-4a02fe6551f8'} Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:19.028881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.030368 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.032314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Returning 404 to user: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:19.032314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8f677e30-b013-42e9-90d6-3439383d74eb tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 404 len: 111 microversion: 2.20 time: 0.031540 Oct 13 04:28:19.032314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 463/987] 10.222.0.27 () {60 vars in 1280 bytes} [Thu Oct 13 04:28:18 2022] DELETE /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:28:19.040983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:19.041595 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] [instance: cc378324-badd-4a63-ba04-4a02fe6551f8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:19.048660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:19.049213 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:19.049752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:19.050583 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:19.051179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:19.051639 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:19.066037 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] HTTP exception thrown: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Request method failure captured: Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 HTTP/1.1 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I1cSk8hX3JU_WckuMYN6y60NIhVtGTpF96ZJ00QD5DtLllxQHaJ8R-i05HsPJElOW6QbgdwYhmTtXPrdAxugZ1ndGxeGo8Zr5D2Ef2fcmlG0zmhRrMJz9O1gXDAcU7KNfBOv9tS97zox4jQeP-5hUoCp4lCJTvUnwozSUO8L5hUF4 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Openstack-Nova-Api-Version: 2.20 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:19.067759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/75387c76ea384e5c96a336751c613e6e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_75387c76ea384e5c96a336751c613e6e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 75387c76ea384e5c96a336751c613e6e Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1606444339 Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 35a058c849104a9699ecfcd0c81d398e Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1606444339-project Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.069456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'cc378324-badd-4a63-ba04-4a02fe6551f8'} Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:19.074814 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. Oct 13 04:28:19.076843 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.078251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] Returning 404 to user: Instance cc378324-badd-4a63-ba04-4a02fe6551f8 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:19.078251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-913ba4e8-dcd4-4370-b4c4-5faa439152fa tempest-AttachVolumeShelveTestJSON-1606444339 tempest-AttachVolumeShelveTestJSON-1606444339-project] 10.222.0.27 "GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8" status: 404 len: 111 microversion: 2.20 time: 0.033523 Oct 13 04:28:19.078251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 525/988] 10.222.0.27 () {60 vars in 1277 bytes} [Thu Oct 13 04:28:19 2022] GET /compute/v2.1/servers/cc378324-badd-4a63-ba04-4a02fe6551f8 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 13 04:28:19.964372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:19.965007 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:19.970986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:19.971475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:19.971915 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:19.972605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:19.973059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:19.973662 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:19.989086 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] HTTP exception thrown: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Request method failure captured: Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 HTTP/1.1 Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I2HLU7FfvJKR98uwPjziNoRi-plQYU_qa31AUawJJfQTDe0ClpEi_AolcXzaGTrMWjpZg5jnnUmsfHEhTDA64zZTOxf6vR_6v7pZrQ5a3ydCzLvSN-EpnkKygAetOOpdbebnBRjk4j66fWOy2g5-ZRtt7NnJKkv9ExzynEwDipG9o Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:19.990960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_73d6c8ffc0354e9d889b1abc471602ac", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 1f6cf725908041ccaceb1ffa694ad161 Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:19.992773 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': '17727b50-e3a3-48f2-a277-7e2f2ff16903'} Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:19.993625 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:19.995099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:19.997023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Returning 404 to user: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:19.997023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f7ba35f7-374e-40dd-b72c-2e58e848c2bb tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 404 len: 111 microversion: 2.1 time: 0.032337 Oct 13 04:28:19.997023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 464/989] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:19 2022] DELETE /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:20.002504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:20.003368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: 17727b50-e3a3-48f2-a277-7e2f2ff16903] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:20.014785 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:20.015282 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:20.015728 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:20.016436 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:20.017087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:20.017512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:20.033627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] HTTP exception thrown: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Request method failure captured: Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 HTTP/1.1 Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I2HLU7FfvJKR98uwPjziNoRi-plQYU_qa31AUawJJfQTDe0ClpEi_AolcXzaGTrMWjpZg5jnnUmsfHEhTDA64zZTOxf6vR_6v7pZrQ5a3ydCzLvSN-EpnkKygAetOOpdbebnBRjk4j66fWOy2g5-ZRtt7NnJKkv9ExzynEwDipG9o Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:20.035214 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_73d6c8ffc0354e9d889b1abc471602ac", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 1f6cf725908041ccaceb1ffa694ad161 Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:20.036613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '17727b50-e3a3-48f2-a277-7e2f2ff16903'} Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:20.037564 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. Oct 13 04:28:20.039972 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:20.042584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Returning 404 to user: Instance 17727b50-e3a3-48f2-a277-7e2f2ff16903 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:20.042584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7f4bffc3-deb3-4a91-a3fe-8f35d9547708 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903" status: 404 len: 111 microversion: 2.1 time: 0.038672 Oct 13 04:28:20.042584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 526/990] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:19 2022] GET /compute/v2.1/servers/17727b50-e3a3-48f2-a277-7e2f2ff16903 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:21.218967 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:21.220159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:21.228950 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:21.229689 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:21.230420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:21.231555 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:21.232845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:21.233280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:21.253612 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] HTTP exception thrown: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Request method failure captured: Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 HTTP/1.1 Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5I2HLU7FfvJKR98uwPjziNoRi-plQYU_qa31AUawJJfQTDe0ClpEi_AolcXzaGTrMWjpZg5jnnUmsfHEhTDA64zZTOxf6vR_6v7pZrQ5a3ydCzLvSN-EpnkKygAetOOpdbebnBRjk4j66fWOy2g5-ZRtt7NnJKkv9ExzynEwDipG9o Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:28:21.260417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_73d6c8ffc0354e9d889b1abc471602ac", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 1f6cf725908041ccaceb1ffa694ad161 Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.262079 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'c318cdef-d4ee-46aa-bc4e-91a49cfe1985'} Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:21.263044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.264576 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.267073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Returning 404 to user: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:21.267958 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e36c3dcc-012d-4b68-81cb-d472b31544f4 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 404 len: 111 microversion: 2.1 time: 0.054573 Oct 13 04:28:21.268840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 465/991] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:21 2022] DELETE /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:21.278435 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:21.279056 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] [instance: c318cdef-d4ee-46aa-bc4e-91a49cfe1985] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:21.286387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:21.287250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:21.289004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:21.289948 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:21.290610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:21.291200 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:21.316313 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] HTTP exception thrown: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Request method failure captured: Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 HTTP/1.1 Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5I2HLU7FfvJKR98uwPjziNoRi-plQYU_qa31AUawJJfQTDe0ClpEi_AolcXzaGTrMWjpZg5jnnUmsfHEhTDA64zZTOxf6vR_6v7pZrQ5a3ydCzLvSN-EpnkKygAetOOpdbebnBRjk4j66fWOy2g5-ZRtt7NnJKkv9ExzynEwDipG9o Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:21.319141 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/73d6c8ffc0354e9d889b1abc471602ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_73d6c8ffc0354e9d889b1abc471602ac", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 73d6c8ffc0354e9d889b1abc471602ac Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1968929909 Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 1f6cf725908041ccaceb1ffa694ad161 Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-AttachVolumeTestJSON-1968929909-project Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.320955 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'c318cdef-d4ee-46aa-bc4e-91a49cfe1985'} Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:21.321900 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. Oct 13 04:28:21.323431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:21.327048 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] Returning 404 to user: Instance c318cdef-d4ee-46aa-bc4e-91a49cfe1985 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:21.328308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-088ee4ae-a5d8-4f06-a57a-ec2b760f2ee8 tempest-AttachVolumeTestJSON-1968929909 tempest-AttachVolumeTestJSON-1968929909-project] 10.222.0.27 "GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985" status: 404 len: 111 microversion: 2.1 time: 0.054480 Oct 13 04:28:21.332906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 527/992] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:21 2022] GET /compute/v2.1/servers/c318cdef-d4ee-46aa-bc4e-91a49cfe1985 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:27.696838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:27.697323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:27.708749 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:27.709668 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:27.713162 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:27.714120 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:27.714785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:27.715426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:27.717880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-62e6f7d6-fa0b-42ef-b033-2dad6e26b91a req-0f6cdf04-ece7-4de4-a919-278f96b652f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:28:27.727934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:28:27.741743 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-62e6f7d6-fa0b-42ef-b033-2dad6e26b91a req-0f6cdf04-ece7-4de4-a919-278f96b652f2 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:27.743030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-62e6f7d6-fa0b-42ef-b033-2dad6e26b91a req-0f6cdf04-ece7-4de4-a919-278f96b652f2 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:27.744252 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-62e6f7d6-fa0b-42ef-b033-2dad6e26b91a req-0f6cdf04-ece7-4de4-a919-278f96b652f2 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:27.751802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:28:27.766683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-62e6f7d6-fa0b-42ef-b033-2dad6e26b91a req-0f6cdf04-ece7-4de4-a919-278f96b652f2 service nova] Creating event network-changed:9b98fb07-be4c-4add-b232-a8149b2588d6 for instance f186151e-3b9a-4cdf-8d6e-427fea09de90 on hv-cinder-86099 Oct 13 04:28:27.772257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-62e6f7d6-fa0b-42ef-b033-2dad6e26b91a req-0f6cdf04-ece7-4de4-a919-278f96b652f2 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058278 Oct 13 04:28:27.772996 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 528/993] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:28:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:27.824091 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:28:27.854855 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:28:27.874822 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-f89e5b55-15ad-424b-a3ee-e1bb840df9ab tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 204 len: 0 microversion: 2.1 time: 0.180968 Oct 13 04:28:27.875785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 466/994] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:28:27 2022] DELETE /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:28:27.887818 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:27.888528 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:27.901161 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:27.901670 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:27.902110 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:27.902860 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:27.906230 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:27.906692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:28.405158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:28.418266 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e286bc3f-de0a-4f0a-b54b-7a47eb581ca9 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.536617 Oct 13 04:28:28.419505 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 529/995] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:27 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:29.442506 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:29.443243 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:29.452663 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:29.453151 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:29.453584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:29.454304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:29.454760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:29.455182 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:29.790476 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:29.798178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e50271d7-8f7d-4d87-beb4-676e14c981c5 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1570 microversion: 2.1 time: 0.359962 Oct 13 04:28:29.798760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 467/996] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:29 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1570 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:28:30.207788 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-b24eff91-4f20-472c-90b8-03204710c895 req-a5ff7a3e-4fd1-4ff4-aff3-0494e6857503 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f186151e-3b9a-4cdf-8d6e-427fea09de90", "name": "network-vif-deleted", "tag": "9b98fb07-be4c-4add-b232-a8149b2588d6"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:28:30.218906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b24eff91-4f20-472c-90b8-03204710c895 req-a5ff7a3e-4fd1-4ff4-aff3-0494e6857503 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:30.219427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b24eff91-4f20-472c-90b8-03204710c895 req-a5ff7a3e-4fd1-4ff4-aff3-0494e6857503 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:30.219864 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-b24eff91-4f20-472c-90b8-03204710c895 req-a5ff7a3e-4fd1-4ff4-aff3-0494e6857503 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:30.232061 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-b24eff91-4f20-472c-90b8-03204710c895 req-a5ff7a3e-4fd1-4ff4-aff3-0494e6857503 service nova] Creating event network-vif-deleted:9b98fb07-be4c-4add-b232-a8149b2588d6 for instance f186151e-3b9a-4cdf-8d6e-427fea09de90 on hv-cinder-86099 Oct 13 04:28:30.236744 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-b24eff91-4f20-472c-90b8-03204710c895 req-a5ff7a3e-4fd1-4ff4-aff3-0494e6857503 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032727 Oct 13 04:28:30.237973 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 530/997] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:28:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:28:30.823145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:30.824224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:30.831067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:30.831878 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:30.832527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:30.833316 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:30.833777 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:30.834224 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:30.867836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:30.868326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:30.868763 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:30.869205 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lazy-loading 'fault' on Instance uuid f186151e-3b9a-4cdf-8d6e-427fea09de90 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:28:30.912032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '242490d937324b31bc95ae1982ea0b39', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fabffb6796a34773987a4f7e414352d0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:28:30.919816 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c3509bee-e2b6-4819-960e-4fe73d2c274b tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 200 len: 1224 microversion: 2.1 time: 0.101423 Oct 13 04:28:30.920406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 468/998] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:30 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:28:31.944422 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:31.945387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] [instance: f186151e-3b9a-4cdf-8d6e-427fea09de90] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:28:31.954053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:31.954823 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:31.955649 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:31.956558 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:28:31.957167 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:28:31.957658 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:28:31.973098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] HTTP exception thrown: Instance f186151e-3b9a-4cdf-8d6e-427fea09de90 could not be found. Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Request method failure captured: Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 HTTP/1.1 Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Km3uIRzlrn47ER99a1TlJ_q-P5K1lJFeKtaPiQVZmVHlAe5BD13Yeh3xpkUqSGypJfaSjABdH02b5WjItVz28R3yfqReQYh7pbWB_MWH8AmtpFHFNqrrJGoyOACAcAAhRRXnlrKjNO2UjYodR35y9iTodzi8fnl5udjSSAfOLp5us Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: fabffb6796a34773987a4f7e414352d0 Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestShelveInstance-731044265 Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:28:31.975848 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fabffb6796a34773987a4f7e414352d0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fabffb6796a34773987a4f7e414352d0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fabffb6796a34773987a4f7e414352d0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fabffb6796a34773987a4f7e414352d0", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestShelveInstance-731044265 Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: fabffb6796a34773987a4f7e414352d0 Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestShelveInstance-731044265 Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestShelveInstance-731044265-project Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 242490d937324b31bc95ae1982ea0b39 Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestShelveInstance-731044265-project Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance f186151e-3b9a-4cdf-8d6e-427fea09de90 could not be found. Oct 13 04:28:31.977650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'f186151e-3b9a-4cdf-8d6e-427fea09de90'} Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f186151e-3b9a-4cdf-8d6e-427fea09de90 could not be found. Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:28:31.978553 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f186151e-3b9a-4cdf-8d6e-427fea09de90 could not be found. Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f186151e-3b9a-4cdf-8d6e-427fea09de90 could not be found. Oct 13 04:28:31.981217 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:28:31.983070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Returning 404 to user: Instance f186151e-3b9a-4cdf-8d6e-427fea09de90 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:28:31.983070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-739a3cfb-8668-4557-9b73-75ecc4daa067 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90" status: 404 len: 111 microversion: 2.1 time: 0.038617 Oct 13 04:28:31.983070 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 531/999] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:28:31 2022] GET /compute/v2.1/servers/f186151e-3b9a-4cdf-8d6e-427fea09de90 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:28:32.202645 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9433f177-fdcc-4f3f-b1fe-a96376bc70f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] Calling method '.version_select at 0x7f07948d8c10>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:28:32.227540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9433f177-fdcc-4f3f-b1fe-a96376bc70f8 tempest-TestShelveInstance-731044265 tempest-TestShelveInstance-731044265-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-243333657" status: 202 len: 0 microversion: 2.1 time: 0.036960 Oct 13 04:28:32.228444 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 469/1000] 10.222.0.27 () {58 vars in 1251 bytes} [Thu Oct 13 04:28:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-243333657 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:29:00.460027 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1867849874", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "networks": [{"uuid": "460e768d-a024-49e4-a4f3-90ecc235b7e9"}]}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:29:00.470171 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=7 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:00.470876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=47 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:00.518476 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] validate_networks() for [('460e768d-a024-49e4-a4f3-90ecc235b7e9', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:29:01.095863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:29:01.096431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:29:01.096917 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:29:01.097346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:29:01.097762 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:29:01.116687 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:01.117199 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:01.117726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:01.118587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:01.119064 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:01.119648 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:01.143661 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Getting quotas for project 7f78c358e5c0423d9ba9520c90f163b5. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:29:01.147929 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Getting quotas for user 4efdcf15e0e448e2a8d651555f4de29a and project 7f78c358e5c0423d9ba9520c90f163b5. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:29:01.160213 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:29:01.162491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:29:01.212466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9921a295-c30a-4a57-882c-e142b9ebcc96 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.756502 Oct 13 04:29:01.213423 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 532/1001] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:29:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 758 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:29:01.222968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=14 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:01.224280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:01.229750 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-552091bf-5ab9-468a-b0b0-c61983a91ccd tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:01.231700 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-552091bf-5ab9-468a-b0b0-c61983a91ccd tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:01.292170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-552091bf-5ab9-468a-b0b0-c61983a91ccd tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:01.300817 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-552091bf-5ab9-468a-b0b0-c61983a91ccd tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1219 microversion: 2.1 time: 0.080092 Oct 13 04:29:01.318076 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 470/1002] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:01 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:02.335250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:02.335868 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:02.344660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:02.345215 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:02.345648 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:02.346406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:02.346856 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:02.347275 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:02.444298 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:02.455573 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5438f680-7798-4d8a-99fd-6126672ec529 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1279 microversion: 2.1 time: 0.126129 Oct 13 04:29:02.457023 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 533/1003] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:02 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1279 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:03.489804 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:03.490439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:03.496970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:03.497450 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:03.497883 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:03.498610 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:03.499068 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:03.499493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:03.631245 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:03.638991 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c76c7ec3-33fd-4112-b019-c550afab0e00 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.153098 Oct 13 04:29:03.639557 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 471/1004] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:03 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:04.662114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:04.662759 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:04.669256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:04.669737 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:04.670177 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:04.671171 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:04.671637 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:04.672065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:04.802918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:04.812006 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a313d306-0f7d-43fb-bec5-33f7be7d788e tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.153688 Oct 13 04:29:04.812578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 534/1005] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:04 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:05.834639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:05.835979 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:05.843776 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:05.844260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:05.844984 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:05.845725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:05.846169 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:05.846807 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:06.080048 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:06.088021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-49f5a405-670d-4133-809a-3c696411e03d tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.257384 Oct 13 04:29:06.088677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 472/1006] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:05 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:06.304068 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-9ecd3b1a-9f35-40d1-9c0a-0866339110f8 req-5df2288d-1c20-4d90-8993-f225dafc1254 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7003e680-8280-4f51-9a15-7fefeb04fae8", "tag": "996bea67-5a64-41cc-903a-6f5823fa4a7f"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:29:06.316153 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9ecd3b1a-9f35-40d1-9c0a-0866339110f8 req-5df2288d-1c20-4d90-8993-f225dafc1254 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:06.316495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9ecd3b1a-9f35-40d1-9c0a-0866339110f8 req-5df2288d-1c20-4d90-8993-f225dafc1254 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:06.316935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-9ecd3b1a-9f35-40d1-9c0a-0866339110f8 req-5df2288d-1c20-4d90-8993-f225dafc1254 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:06.328278 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-9ecd3b1a-9f35-40d1-9c0a-0866339110f8 req-5df2288d-1c20-4d90-8993-f225dafc1254 service nova] Creating event network-changed:996bea67-5a64-41cc-903a-6f5823fa4a7f for instance 7003e680-8280-4f51-9a15-7fefeb04fae8 on hv-cinder-86099 Oct 13 04:29:06.333073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-9ecd3b1a-9f35-40d1-9c0a-0866339110f8 req-5df2288d-1c20-4d90-8993-f225dafc1254 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032723 Oct 13 04:29:06.333852 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 535/1007] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:29:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:29:07.111905 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:07.112552 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:07.118920 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:07.119401 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:07.119835 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:07.120519 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:07.121797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:07.123221 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:07.260826 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:07.268614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a7725d38-d178-41c1-afe0-ee53ab529a3a tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.160521 Oct 13 04:29:07.269170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 473/1008] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:07 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:08.293440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:08.294179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:08.300406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:08.300886 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:08.301321 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:08.302010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:08.302563 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:08.302986 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:08.434162 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:08.441885 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ec5cb4e0-6db7-4e8b-b16b-849aa52ced98 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.153828 Oct 13 04:29:08.442486 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 536/1009] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:08 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:09.465908 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:09.466536 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:09.473411 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:09.473947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:09.474399 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:09.475149 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:09.475645 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:09.476070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:09.607707 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:09.615325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-576082c4-0e40-42f9-b1e0-d8069f340cea tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.154122 Oct 13 04:29:09.615886 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 474/1010] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:09 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:29:10.638132 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:10.638710 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:10.644999 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:10.645481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:10.645923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:10.646650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:10.647104 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:10.647530 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:10.782537 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:10.790772 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-749e135b-e629-4b9d-bbc2-950784e702ba tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.156717 Oct 13 04:29:10.791826 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 537/1011] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:10 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:11.816832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:11.817535 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:11.820985 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:11.826232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:11.826849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:11.827359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:11.828135 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:11.828676 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:11.829172 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:11.972944 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:11.981513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7cb02af4-6436-462d-bef0-28466a7d93eb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.168957 Oct 13 04:29:11.982341 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 475/1012] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:11 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:13.004979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:13.005653 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:13.007636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:13.008525 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=6 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:13.014912 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:13.015861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:13.016641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:13.018535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:13.019124 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:13.019774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:13.167030 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:13.175432 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-aeafbf0a-a372-4f3c-ace8-4b4c3a18b880 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.174595 Oct 13 04:29:13.176007 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 538/1013] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:12 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:13.471906 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-e60b7f0f-759a-4235-b927-414255a2fc1a req-847f6ed7-5037-473c-8b49-ff4b18403ba1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7003e680-8280-4f51-9a15-7fefeb04fae8", "name": "network-vif-plugged", "status": "completed", "tag": "996bea67-5a64-41cc-903a-6f5823fa4a7f"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:29:13.483697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-e60b7f0f-759a-4235-b927-414255a2fc1a req-847f6ed7-5037-473c-8b49-ff4b18403ba1 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:13.484637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-e60b7f0f-759a-4235-b927-414255a2fc1a req-847f6ed7-5037-473c-8b49-ff4b18403ba1 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:13.485229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-e60b7f0f-759a-4235-b927-414255a2fc1a req-847f6ed7-5037-473c-8b49-ff4b18403ba1 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:13.500295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-e60b7f0f-759a-4235-b927-414255a2fc1a req-847f6ed7-5037-473c-8b49-ff4b18403ba1 service nova] Creating event network-vif-plugged:996bea67-5a64-41cc-903a-6f5823fa4a7f for instance 7003e680-8280-4f51-9a15-7fefeb04fae8 on hv-cinder-86099 Oct 13 04:29:13.505873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-e60b7f0f-759a-4235-b927-414255a2fc1a req-847f6ed7-5037-473c-8b49-ff4b18403ba1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038082 Oct 13 04:29:13.506642 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 476/1014] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:29:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:29:14.200343 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:14.201659 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:14.216366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:14.217201 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:14.218378 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:14.220442 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:14.222072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:14.222546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:14.421894 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:14.430304 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f9c4e47a-c64a-48e5-a410-40253b1a0465 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1319 microversion: 2.1 time: 0.235262 Oct 13 04:29:14.431166 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 539/1015] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:14 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1319 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:15.456275 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:29:15.456970 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:15.463611 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:15.464699 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:15.465158 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:15.465867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:15.466349 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:15.467032 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:15.619864 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:29:15.628194 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e1054bec-93a1-4014-8b15-f81757f4f095 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1508 microversion: 2.1 time: 0.176852 Oct 13 04:29:15.628768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 477/1016] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:29:15 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1508 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:29:37.349299 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "aad29b60-4097-4f9f-b2af-073182dc49d2", "device": "/dev/sdb"}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:29:37.352211 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:29:37.354563 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:37.355431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=40 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:37.361975 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:37.362489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:37.362928 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:37.363614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:29:37.364059 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:29:37.364482 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:29:39.963561 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-42379e64-1c14-483f-ae7a-eec3e7fd1e37 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "POST /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.618052 Oct 13 04:29:39.964229 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 540/1017] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:29:37 2022] POST /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8/os-volume_attachments => generated 194 bytes in 2620 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:29:57.270884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=18 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:57.271814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=38 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:29:57.275008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 13 04:29:57.482232 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-fcaedc95-cb0b-4d0e-a222-aba3690d6c5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Action: 'create', calling method: .version_select at 0x7f0794a84af0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-22921279", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFoCemLNUrI493sDZcBvowfGbD+tE35zOOpjRrjz27oInmfmPsrO2woaMxRvnMCAv6fatLu0yfvepCSxZch6a+8FogoTpJMolUvhSD0vEMHeZbu9UgmKOeNLFZFvu3L2FA=="}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:29:57.497589 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-fcaedc95-cb0b-4d0e-a222-aba3690d6c5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:29:57.502168 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-fcaedc95-cb0b-4d0e-a222-aba3690d6c5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for user 4cd203fe04644ace8be512ea2c455774 and project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'key_pairs'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:29:57.525186 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-fcaedc95-cb0b-4d0e-a222-aba3690d6c5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.260442 Oct 13 04:29:57.525783 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 478/1018] 10.222.0.27 () {60 vars in 1159 bytes} [Thu Oct 13 04:29:57 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:30:12.458938 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:12.459480 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:12.461967 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:12.462809 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:12.471226 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:12.471744 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:12.472182 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:12.472913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:12.473368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:12.473790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:12.558862 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.objects.instance [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lazy-loading 'services' on Instance uuid 7003e680-8280-4f51-9a15-7fefeb04fae8 {{(pid=118673) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:30:12.634691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9c9a2689-d93e-46e0-b2af-ed8814437409 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "DELETE /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8/os-volume_attachments/aad29b60-4097-4f9f-b2af-073182dc49d2" status: 202 len: 0 microversion: 2.1 time: 0.179946 Oct 13 04:30:12.635317 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 541/1019] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:30:12 2022] DELETE /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8/os-volume_attachments/aad29b60-4097-4f9f-b2af-073182dc49d2 => generated 0 bytes in 182 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:30:13.777655 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=4 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:13.778416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:13.783347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:13.783838 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:13.791948 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:13.792427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:13.792858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:13.793540 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:13.793988 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:13.794447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:13.820639 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:30:13.898790 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-0af5e3a6-fa07-40cc-b6c3-065c2f50d0bb tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "DELETE /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 204 len: 0 microversion: 2.1 time: 0.122028 Oct 13 04:30:13.899402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 479/1020] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:30:13 2022] DELETE /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:30:13.910455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:13.911154 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:13.917777 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:13.918401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:13.918847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:13.919636 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:13.920145 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:13.920597 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:14.062225 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:14.070096 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9b8a4518-0c9f-4913-be10-e7b97011a2f8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1514 microversion: 2.1 time: 0.166121 Oct 13 04:30:14.070699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 542/1021] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:13 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1514 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:15.094325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:15.095098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:15.103809 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:15.104410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:15.104865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:15.105588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:15.106426 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:15.106898 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:15.244723 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:15.252880 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-70cbd705-9b9b-4b57-9b50-368477062819 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1514 microversion: 2.1 time: 0.163475 Oct 13 04:30:15.253437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 480/1022] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:15 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1514 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:16.201839 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-c9be1a0a-e5d1-4736-833b-5496f75991a8 req-13aae267-9f07-4d16-8cf6-5f1276c7d075 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7003e680-8280-4f51-9a15-7fefeb04fae8", "name": "network-vif-deleted", "tag": "996bea67-5a64-41cc-903a-6f5823fa4a7f"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:16.212896 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c9be1a0a-e5d1-4736-833b-5496f75991a8 req-13aae267-9f07-4d16-8cf6-5f1276c7d075 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:16.213368 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c9be1a0a-e5d1-4736-833b-5496f75991a8 req-13aae267-9f07-4d16-8cf6-5f1276c7d075 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:16.213983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-c9be1a0a-e5d1-4736-833b-5496f75991a8 req-13aae267-9f07-4d16-8cf6-5f1276c7d075 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:16.226633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-c9be1a0a-e5d1-4736-833b-5496f75991a8 req-13aae267-9f07-4d16-8cf6-5f1276c7d075 service nova] Creating event network-vif-deleted:996bea67-5a64-41cc-903a-6f5823fa4a7f for instance 7003e680-8280-4f51-9a15-7fefeb04fae8 on hv-cinder-86099 Oct 13 04:30:16.231785 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-c9be1a0a-e5d1-4736-833b-5496f75991a8 req-13aae267-9f07-4d16-8cf6-5f1276c7d075 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033535 Oct 13 04:30:16.236223 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 543/1023] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:30:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:30:16.280465 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:16.281219 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:16.293059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:16.293694 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:16.294737 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:16.295545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:16.296130 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:16.297598 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:16.371342 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:16.380637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-921ed33d-f004-4cb3-a4d5-36563fb76f3c tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 200 len: 1304 microversion: 2.1 time: 0.107708 Oct 13 04:30:16.381725 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 481/1024] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:16 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 1304 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:17.405396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:17.406863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: 7003e680-8280-4f51-9a15-7fefeb04fae8] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:17.413055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:17.413546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:17.413980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:17.414706 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:17.415159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:17.415587 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:17.429672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] HTTP exception thrown: Instance 7003e680-8280-4f51-9a15-7fefeb04fae8 could not be found. Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Request method failure captured: Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 HTTP/1.1 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5QKVO1NN48CzRENvZpwjkKIOZHcei_m60TFRgKl5k4UWfSKs3s0_4hPCQhHRj83Lc2db3nzb1lQVA7a1JJXBKmb2iSlJ7GyobmwazJc7rCsW8nsFVfw3grk6dP7KYXGTguoe4LsLfcGHsyXluZZMjRn6BCOtNtoWEzjtxBZu1TEkz8 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: 7f78c358e5c0423d9ba9520c90f163b5 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1107795008 Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:30:17.431341 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7f78c358e5c0423d9ba9520c90f163b5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7f78c358e5c0423d9ba9520c90f163b5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7f78c358e5c0423d9ba9520c90f163b5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7f78c358e5c0423d9ba9520c90f163b5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-VolumesSnapshotTestJSON-1107795008 Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: 7f78c358e5c0423d9ba9520c90f163b5 Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1107795008 Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-VolumesSnapshotTestJSON-1107795008-project Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 4efdcf15e0e448e2a8d651555f4de29a Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-VolumesSnapshotTestJSON-1107795008-project Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 7003e680-8280-4f51-9a15-7fefeb04fae8 could not be found. Oct 13 04:30:17.433055 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '7003e680-8280-4f51-9a15-7fefeb04fae8'} Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7003e680-8280-4f51-9a15-7fefeb04fae8 could not be found. Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:30:17.434308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7003e680-8280-4f51-9a15-7fefeb04fae8 could not be found. Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7003e680-8280-4f51-9a15-7fefeb04fae8 could not be found. Oct 13 04:30:17.435913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:30:17.437494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Returning 404 to user: Instance 7003e680-8280-4f51-9a15-7fefeb04fae8 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:30:17.437494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-82152b1a-0376-4812-bb70-88a47cf391ab tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8" status: 404 len: 111 microversion: 2.1 time: 0.031998 Oct 13 04:30:17.437494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 544/1025] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:17 2022] GET /compute/v2.1/servers/7003e680-8280-4f51-9a15-7fefeb04fae8 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:30:23.455674 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-847931657", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-22921279", "security_groups": [{"name": "tempest-secgroup-smoke-54814354"}], "block_device_mapping_v2": [{"uuid": "eaecac9c-8d80-4db6-a5aa-0ec9255a3a1e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "2d3f0d2e-0640-4034-90cf-e88b8e787a37"}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:23.571487 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] validate_networks() for [('2d3f0d2e-0640-4034-90cf-e88b8e787a37', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:30:24.033865 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:30:24.034330 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:30:24.034949 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:30:24.035374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:30:24.035824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:30:24.044565 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:24.045070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:24.045508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:24.046293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:24.046753 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:24.047178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:24.063845 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:30:24.067813 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for user 4cd203fe04644ace8be512ea2c455774 and project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:30:24.123925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:30:24.126833 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='eaecac9c-8d80-4db6-a5aa-0ec9255a3a1e',volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:30:24.261572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1fc1ea4a-a752-4a95-9746-ab260e694a1b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.810135 Oct 13 04:30:24.262319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 482/1026] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:30:23 2022] POST /compute/v2.1/servers => generated 404 bytes in 812 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 13 04:30:24.274040 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-230953a4-5ac9-423c-9ba2-5ab50b3f79cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:24.274672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-230953a4-5ac9-423c-9ba2-5ab50b3f79cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:24.329954 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-230953a4-5ac9-423c-9ba2-5ab50b3f79cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:24.336873 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-230953a4-5ac9-423c-9ba2-5ab50b3f79cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1092 microversion: 2.1 time: 0.066658 Oct 13 04:30:24.337440 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 545/1027] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:24 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1092 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:30:25.361256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:25.361873 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:25.368864 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:25.369382 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:25.369818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:25.370549 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:25.371014 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:25.371467 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:25.431975 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:25.440833 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8c14f826-0984-4c22-af10-09fc403f1a52 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1142 microversion: 2.1 time: 0.084474 Oct 13 04:30:25.441403 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 483/1028] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:25 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1142 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:26.300288 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1162713349", "imageRef": "ec445316-161b-45cb-810b-9d5cecc7e897", "flavorRef": "452", "networks": [{"uuid": "460e768d-a024-49e4-a4f3-90ecc235b7e9"}]}} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:26.353979 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.network.neutron [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] validate_networks() for [('460e768d-a024-49e4-a4f3-90ecc235b7e9', None, None, None, None, None)] {{(pid=118673) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:30:26.464427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:26.465072 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:26.476357 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:26.476885 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:26.477297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:26.478008 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:26.478751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:26.479361 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:26.555070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:26.567208 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a6201dad-22c8-439f-aa4c-fa32a1ab3c57 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1198 microversion: 2.1 time: 0.106716 Oct 13 04:30:26.567780 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 484/1029] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:26 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1198 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:30:26.740623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Flavor limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:30:26.741053 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Image limits 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:30:26.741526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Flavor pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:30:26.741935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Image pref 0:0:0 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:30:26.742401 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.virt.hardware [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118673) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:30:26.751976 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:26.752456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:26.752895 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:26.753620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:26.754072 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:26.754535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:26.772303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Getting quotas for project 7f78c358e5c0423d9ba9520c90f163b5. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:30:26.776209 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.quota [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Getting quotas for user 4efdcf15e0e448e2a8d651555f4de29a and project 7f78c358e5c0423d9ba9520c90f163b5. Resources: {'cores', 'ram', 'instances'} {{(pid=118673) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:30:26.787065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Going to run 1 instances... {{(pid=118673) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:30:26.789367 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ec445316-161b-45cb-810b-9d5cecc7e897',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118673) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:30:26.834676 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9baa23ba-02f7-4887-97ef-0264dbe33706 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.538717 Oct 13 04:30:26.835497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 546/1030] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:30:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 540 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:30:26.846709 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-10db0d0b-2184-4977-9f7c-e09ee21be8e9 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:26.847353 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-10db0d0b-2184-4977-9f7c-e09ee21be8e9 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:26.924585 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-10db0d0b-2184-4977-9f7c-e09ee21be8e9 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:26.931533 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-10db0d0b-2184-4977-9f7c-e09ee21be8e9 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1219 microversion: 2.1 time: 0.088885 Oct 13 04:30:26.932059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 485/1031] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:26 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:27.593645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:27.595028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:27.605004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:27.605503 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:27.606221 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:27.606927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:27.607379 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:27.607800 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:27.810128 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:27.821613 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ae7d4b76-3ced-4894-9684-cd6bc373a8c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.230878 Oct 13 04:30:27.822277 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 547/1032] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:27 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:27.955647 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:27.956262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:27.963955 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:27.964421 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:27.964863 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:27.965572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:27.966030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:27.966488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:28.064034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:28.072346 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-787a0e49-820a-4b9b-8699-8c18a41da87f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1271 microversion: 2.1 time: 0.121089 Oct 13 04:30:28.072927 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 486/1033] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:27 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1271 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:28.391392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-bb9a55ec-2215-42b3-912d-c157b9d12c88 req-f5bc923b-993d-44f5-b00a-7ab0c75b2bea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c4106e63-3d42-4f62-8eb2-6b97a94b3c71", "tag": "909e2718-9785-4e2c-ae5c-75201a9f4fc5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:28.403198 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-bb9a55ec-2215-42b3-912d-c157b9d12c88 req-f5bc923b-993d-44f5-b00a-7ab0c75b2bea service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:28.403691 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-bb9a55ec-2215-42b3-912d-c157b9d12c88 req-f5bc923b-993d-44f5-b00a-7ab0c75b2bea service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:28.404125 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-bb9a55ec-2215-42b3-912d-c157b9d12c88 req-f5bc923b-993d-44f5-b00a-7ab0c75b2bea service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:28.426456 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-bb9a55ec-2215-42b3-912d-c157b9d12c88 req-f5bc923b-993d-44f5-b00a-7ab0c75b2bea service nova] Creating event network-changed:909e2718-9785-4e2c-ae5c-75201a9f4fc5 for instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 on hv-cinder-86099 Oct 13 04:30:28.432920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-bb9a55ec-2215-42b3-912d-c157b9d12c88 req-f5bc923b-993d-44f5-b00a-7ab0c75b2bea service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045707 Oct 13 04:30:28.433887 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 548/1034] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:30:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:30:28.846534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:28.847636 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:28.854436 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:28.855012 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:28.855446 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:28.856410 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:28.856862 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:28.857279 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:29.098941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:29.099882 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:29.107892 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:29.109103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:29.109607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:29.110481 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:29.111011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:29.111661 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:29.192311 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:29.201850 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-edb916a2-ccc7-43a7-a665-be4cc2b6f169 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1277 microversion: 2.1 time: 0.108189 Oct 13 04:30:29.202567 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 549/1035] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:29 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1277 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:30:29.453897 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:29.463170 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-59e179d6-dd7d-4aff-b0a7-7f4bf723a395 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.621989 Oct 13 04:30:29.463756 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 487/1036] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:28 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:30.225314 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:30.225923 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:30.235429 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:30.235918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:30.236349 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:30.237049 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:30.237647 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:30.238071 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:30.389035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:30.400455 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7ec49d19-c4e4-4478-8249-57b683dea6e0 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1319 microversion: 2.1 time: 0.179002 Oct 13 04:30:30.410916 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 550/1037] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:30 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1319 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:30.488511 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:30.489248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:30.495717 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:30.496523 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:30.497095 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:30.497884 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:30.498475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:30.499333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:30.815592 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:30.828939 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-1398a660-ef34-40c4-80e4-95e0cfbec7d1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.344306 Oct 13 04:30:30.829531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 488/1038] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:30 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:30.973489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-f68df2e5-8d79-43c9-8329-1303a81f1ec9 req-d1995285-3b5b-4268-b64d-2c05ed030fe2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bf80e953-e21c-4f85-a8fe-e0377896f0a1", "tag": "810c2a37-62ae-4bb5-8a08-e8b2b26033a4"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:30.988444 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-f68df2e5-8d79-43c9-8329-1303a81f1ec9 req-d1995285-3b5b-4268-b64d-2c05ed030fe2 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:30.989173 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-f68df2e5-8d79-43c9-8329-1303a81f1ec9 req-d1995285-3b5b-4268-b64d-2c05ed030fe2 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:30.989638 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-f68df2e5-8d79-43c9-8329-1303a81f1ec9 req-d1995285-3b5b-4268-b64d-2c05ed030fe2 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:31.005002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-f68df2e5-8d79-43c9-8329-1303a81f1ec9 req-d1995285-3b5b-4268-b64d-2c05ed030fe2 service nova] Creating event network-changed:810c2a37-62ae-4bb5-8a08-e8b2b26033a4 for instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 on hv-cinder-86099 Oct 13 04:30:31.009861 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-f68df2e5-8d79-43c9-8329-1303a81f1ec9 req-d1995285-3b5b-4268-b64d-2c05ed030fe2 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041739 Oct 13 04:30:31.012116 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 551/1039] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:30:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:30:31.424132 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:31.424752 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:31.430948 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:31.431442 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:31.431877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:31.432588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:31.433553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:31.433553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:31.569469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:31.578733 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-db70474f-48cf-494d-8700-b49457567af1 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1319 microversion: 2.1 time: 0.159023 Oct 13 04:30:31.579307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 489/1040] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:31 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:31.854000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:31.854791 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:31.862139 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:31.862658 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:31.863093 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:31.863798 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:31.864310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:31.864730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:32.004184 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:32.012551 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4af6cdd6-48d5-4912-9ebe-bb2e9cbffd59 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.163281 Oct 13 04:30:32.013139 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 552/1041] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:31 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:32.602369 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:32.603139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:32.610315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:32.610811 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:32.611251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:32.612099 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:32.612744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:32.613080 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:32.756532 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:32.764242 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-28fcfb41-5d86-47f6-a207-7ae9e948dc32 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1319 microversion: 2.1 time: 0.165833 Oct 13 04:30:32.765364 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 490/1042] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:32 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:33.036272 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:33.036810 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:33.043112 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:33.043593 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:33.044024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:33.044730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:33.045164 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:33.045582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:33.186236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:33.194616 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-92596203-8dca-427d-a281-f4a9b0120cdc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.162258 Oct 13 04:30:33.195285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 553/1043] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:33 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:33.787504 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:33.788723 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:33.796850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:33.797347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:33.797782 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:33.798527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:33.798981 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:33.799404 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:33.941693 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:33.949666 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e7e8d4bd-c987-4fbd-a618-76e15ef9ddcc tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1319 microversion: 2.1 time: 0.166082 Oct 13 04:30:33.950267 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 491/1044] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:33 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1319 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:34.217446 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:34.218970 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:34.227258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:34.227743 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:34.228201 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:34.228956 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:34.229497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:34.229919 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:34.384035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:34.393539 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0b4e99bb-36dc-4ea0-8700-e3275098ce44 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.180043 Oct 13 04:30:34.394126 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 554/1045] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:34 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:34.972960 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:34.973493 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:34.975588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:34.981799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:34.982315 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:34.982748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:34.983990 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:34.984509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:34.984955 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:35.116710 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:35.124982 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-3bae00e4-6a2d-41ad-af91-456b0c04a382 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1319 microversion: 2.1 time: 0.155992 Oct 13 04:30:35.125553 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 492/1046] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:34 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:30:35.416903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:35.417600 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:35.424494 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:35.424977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:35.425414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:35.426111 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:35.426591 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:35.427018 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:35.558153 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:35.566232 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-466c43b4-7e38-4a2c-b7a4-fad82c3f75fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.153241 Oct 13 04:30:35.566794 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 555/1047] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:35 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:36.050881 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-10fd6110-b695-4c67-976c-557dd5d39365 req-25adf2c9-1477-4a40-8daf-4f18560f71de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bf80e953-e21c-4f85-a8fe-e0377896f0a1", "name": "network-vif-plugged", "status": "completed", "tag": "810c2a37-62ae-4bb5-8a08-e8b2b26033a4"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:36.063148 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-10fd6110-b695-4c67-976c-557dd5d39365 req-25adf2c9-1477-4a40-8daf-4f18560f71de service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:36.063607 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-10fd6110-b695-4c67-976c-557dd5d39365 req-25adf2c9-1477-4a40-8daf-4f18560f71de service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:36.064033 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-10fd6110-b695-4c67-976c-557dd5d39365 req-25adf2c9-1477-4a40-8daf-4f18560f71de service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:36.075287 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-10fd6110-b695-4c67-976c-557dd5d39365 req-25adf2c9-1477-4a40-8daf-4f18560f71de service nova] Creating event network-vif-plugged:810c2a37-62ae-4bb5-8a08-e8b2b26033a4 for instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 on hv-cinder-86099 Oct 13 04:30:36.081702 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-10fd6110-b695-4c67-976c-557dd5d39365 req-25adf2c9-1477-4a40-8daf-4f18560f71de service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034210 Oct 13 04:30:36.082648 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 493/1048] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:30:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:30:36.149504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:36.150486 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:36.157905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:36.158420 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:36.158876 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:36.159559 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:36.160063 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:36.160568 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:36.296079 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:36.304060 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-7b1cbca5-58fd-405a-829a-8ecfe264436b tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1319 microversion: 2.1 time: 0.158443 Oct 13 04:30:36.304633 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 556/1049] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:36 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:36.589487 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:36.590095 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:36.597651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:36.598137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:36.598593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:36.599278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:36.599729 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:36.600143 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:36.734899 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:36.744054 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a329b1e9-06b2-4898-b509-3cac5f782d51 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.158790 Oct 13 04:30:36.744628 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 494/1050] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:36 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:37.326335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:37.327126 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:37.329562 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:37.330805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=8 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:37.337930 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:37.338937 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:37.339225 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:37.339863 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:37.340283 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:37.341118 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:37.472412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:37.480101 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-60e62352-438a-427d-a171-1e3405a994d2 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1508 microversion: 2.1 time: 0.157680 Oct 13 04:30:37.480932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 557/1051] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:37 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1508 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:37.504376 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "77aa69c5-c66c-4ec0-840d-09269ee610e6", "device": "/dev/sdb"}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:37.507614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:37.509635 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=8 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:37.515861 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:37.516351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:37.516802 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:37.517509 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:37.517943 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:37.518391 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:37.767560 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:37.768292 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:37.774507 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:37.775088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:37.775604 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:37.776376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:37.776914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:37.777412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:37.817431 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:37.930627 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:37.939485 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4ea8cb09-8af3-462d-a33d-63ae414009ee tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.175971 Oct 13 04:30:37.954589 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 558/1052] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:37 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:30:37.964615 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-582f96d9-1335-4e33-990e-181353f68ec8 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "POST /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.464712 Oct 13 04:30:37.965475 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 495/1053] 10.222.0.27 () {60 vars in 1323 bytes} [Thu Oct 13 04:30:37 2022] POST /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1/os-volume_attachments => generated 194 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:30:38.098074 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-2d76157c-43db-4066-b521-888c916ef38f req-e4ae0083-8b3b-40a0-82f8-1a338bc4ceaf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4106e63-3d42-4f62-8eb2-6b97a94b3c71", "name": "network-vif-plugged", "status": "completed", "tag": "909e2718-9785-4e2c-ae5c-75201a9f4fc5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:38.107706 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2d76157c-43db-4066-b521-888c916ef38f req-e4ae0083-8b3b-40a0-82f8-1a338bc4ceaf service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:38.108787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2d76157c-43db-4066-b521-888c916ef38f req-e4ae0083-8b3b-40a0-82f8-1a338bc4ceaf service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:38.109441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-2d76157c-43db-4066-b521-888c916ef38f req-e4ae0083-8b3b-40a0-82f8-1a338bc4ceaf service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:38.120779 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-2d76157c-43db-4066-b521-888c916ef38f req-e4ae0083-8b3b-40a0-82f8-1a338bc4ceaf service nova] Creating event network-vif-plugged:909e2718-9785-4e2c-ae5c-75201a9f4fc5 for instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 on hv-cinder-86099 Oct 13 04:30:38.126768 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-2d76157c-43db-4066-b521-888c916ef38f req-e4ae0083-8b3b-40a0-82f8-1a338bc4ceaf service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032367 Oct 13 04:30:38.127351 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 559/1054] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:30:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:30:38.964178 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:38.964789 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:38.972187 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:38.972663 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:38.973092 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:38.973772 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:38.974233 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:38.974655 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:39.109447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:39.117206 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-26379788-ca29-40e8-bd2b-6d0c3e9fc5b1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1262 microversion: 2.1 time: 0.157352 Oct 13 04:30:39.117785 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 496/1055] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:38 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1262 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:40.142635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:40.143333 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:40.149616 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:40.150098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:40.150557 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:40.151245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:40.151692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:40.152108 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:40.281989 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:40.289630 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-a817a294-b8d5-48fa-b451-752ff57afc82 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1450 microversion: 2.1 time: 0.153214 Oct 13 04:30:40.290188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 560/1056] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:40 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1450 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:40.311993 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:40.312605 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:40.318969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:40.319454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:40.319891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:40.320572 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:40.321028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:40.321447 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:40.453755 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:40.461454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-57be2293-533e-463e-be69-acda53281fb9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1450 microversion: 2.1 time: 0.153261 Oct 13 04:30:40.462021 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 497/1057] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:40 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1450 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:30:42.111360 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-57bb346a-da6a-4432-90f9-6a5523379921 req-4fb8c47a-33fa-46d2-a9ea-fa5513c6e561 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c4106e63-3d42-4f62-8eb2-6b97a94b3c71", "tag": "909e2718-9785-4e2c-ae5c-75201a9f4fc5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:30:42.120944 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-57bb346a-da6a-4432-90f9-6a5523379921 req-4fb8c47a-33fa-46d2-a9ea-fa5513c6e561 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:42.121417 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-57bb346a-da6a-4432-90f9-6a5523379921 req-4fb8c47a-33fa-46d2-a9ea-fa5513c6e561 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:42.121841 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-57bb346a-da6a-4432-90f9-6a5523379921 req-4fb8c47a-33fa-46d2-a9ea-fa5513c6e561 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:42.133787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-57bb346a-da6a-4432-90f9-6a5523379921 req-4fb8c47a-33fa-46d2-a9ea-fa5513c6e561 service nova] Creating event network-changed:909e2718-9785-4e2c-ae5c-75201a9f4fc5 for instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 on hv-cinder-86099 Oct 13 04:30:42.138951 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-57bb346a-da6a-4432-90f9-6a5523379921 req-4fb8c47a-33fa-46d2-a9ea-fa5513c6e561 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032090 Oct 13 04:30:42.140293 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 561/1058] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:30:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:30:56.307508 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:56.308002 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:56.310651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:56.311969 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=16,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:56.318751 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:56.319239 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:56.319671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:56.320366 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:56.320818 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:56.321241 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:56.558063 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lazy-loading 'services' on Instance uuid bf80e953-e21c-4f85-a8fe-e0377896f0a1 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:30:56.630145 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-d108f947-d707-414f-91ee-a2510066f64f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "DELETE /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1/os-volume_attachments/77aa69c5-c66c-4ec0-840d-09269ee610e6" status: 202 len: 0 microversion: 2.1 time: 0.326315 Oct 13 04:30:56.630814 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 498/1059] 10.222.0.27 () {58 vars in 1416 bytes} [Thu Oct 13 04:30:56 2022] DELETE /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1/os-volume_attachments/77aa69c5-c66c-4ec0-840d-09269ee610e6 => generated 0 bytes in 328 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 13 04:30:57.754446 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=3 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:57.755310 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:30:57.759632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:57.760213 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:57.768552 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:57.769037 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:57.769471 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:57.770163 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:57.770654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:57.771067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:57.796866 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:30:57.871588 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1dbb65d8-d504-4527-8080-42cf08043715 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "DELETE /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 204 len: 0 microversion: 2.1 time: 0.118441 Oct 13 04:30:57.872199 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 562/1060] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:30:57 2022] DELETE /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:30:57.882405 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:57.883097 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:57.889124 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:57.890241 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:57.891046 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:57.891841 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:57.892308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:57.892711 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:58.021259 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:58.029010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-ff2027bf-c628-4ecb-beb1-56dcb0923019 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1514 microversion: 2.1 time: 0.151319 Oct 13 04:30:58.029587 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 499/1061] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:57 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1514 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:30:59.054450 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:30:59.055065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:30:59.062088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:59.062614 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:59.063051 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:59.063739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:30:59.064193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:30:59.064618 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:30:59.244692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:30:59.252616 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c25e09f6-ea24-489f-9087-12a28cb9fa33 tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1514 microversion: 2.1 time: 0.202545 Oct 13 04:30:59.253188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 563/1062] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:30:59 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1514 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:31:00.205532 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-418fd2bb-97e6-4927-a3c0-5c4e2f66a261 req-6a4cb44b-1a63-4666-825c-b287b496b7dd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bf80e953-e21c-4f85-a8fe-e0377896f0a1", "name": "network-vif-deleted", "tag": "810c2a37-62ae-4bb5-8a08-e8b2b26033a4"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:31:00.215833 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-418fd2bb-97e6-4927-a3c0-5c4e2f66a261 req-6a4cb44b-1a63-4666-825c-b287b496b7dd service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:31:00.216307 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-418fd2bb-97e6-4927-a3c0-5c4e2f66a261 req-6a4cb44b-1a63-4666-825c-b287b496b7dd service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:31:00.216731 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-418fd2bb-97e6-4927-a3c0-5c4e2f66a261 req-6a4cb44b-1a63-4666-825c-b287b496b7dd service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:31:00.228767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-418fd2bb-97e6-4927-a3c0-5c4e2f66a261 req-6a4cb44b-1a63-4666-825c-b287b496b7dd service nova] Creating event network-vif-deleted:810c2a37-62ae-4bb5-8a08-e8b2b26033a4 for instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 on hv-cinder-86099 Oct 13 04:31:00.236406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-418fd2bb-97e6-4927-a3c0-5c4e2f66a261 req-6a4cb44b-1a63-4666-825c-b287b496b7dd service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035252 Oct 13 04:31:00.236925 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 500/1063] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:31:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:31:00.276674 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:31:00.277443 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:31:00.284082 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:31:00.284578 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:31:00.285036 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:31:00.285739 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:31:00.286195 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:31:00.286683 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:31:00.354342 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4efdcf15e0e448e2a8d651555f4de29a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f78c358e5c0423d9ba9520c90f163b5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:31:00.363011 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6bfd1457-a685-4e5f-b435-4dd85e2cfefa tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 200 len: 1304 microversion: 2.1 time: 0.091245 Oct 13 04:31:00.363510 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 564/1064] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:31:00 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 1304 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:31:01.385980 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:31:01.386624 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] [instance: bf80e953-e21c-4f85-a8fe-e0377896f0a1] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:31:01.393314 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:31:01.393797 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:31:01.394256 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:31:01.394951 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:31:01.395406 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:31:01.395833 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:31:01.409862 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] HTTP exception thrown: Instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 could not be found. Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Request method failure captured: Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 HTTP/1.1 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5QKVO1NN48CzRENvZpwjkKIOZHcei_m60TFRgKl5k4UWfSKs3s0_4hPCQhHRj83Lc2db3nzb1lQVA7a1JJXBKmb2iSlJ7GyobmwazJc7rCsW8nsFVfw3grk6dP7KYXGTguoe4LsLfcGHsyXluZZMjRn6BCOtNtoWEzjtxBZu1TEkz8 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: 7f78c358e5c0423d9ba9520c90f163b5 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1107795008 Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:31:01.411571 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7f78c358e5c0423d9ba9520c90f163b5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7f78c358e5c0423d9ba9520c90f163b5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7f78c358e5c0423d9ba9520c90f163b5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7f78c358e5c0423d9ba9520c90f163b5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-VolumesSnapshotTestJSON-1107795008 Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: 7f78c358e5c0423d9ba9520c90f163b5 Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1107795008 Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-VolumesSnapshotTestJSON-1107795008-project Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 4efdcf15e0e448e2a8d651555f4de29a Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-VolumesSnapshotTestJSON-1107795008-project Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 could not be found. Oct 13 04:31:01.413073 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'bf80e953-e21c-4f85-a8fe-e0377896f0a1'} Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 could not be found. Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:31:01.413888 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 could not be found. Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 could not be found. Oct 13 04:31:01.415305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:31:01.416579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] Returning 404 to user: Instance bf80e953-e21c-4f85-a8fe-e0377896f0a1 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:31:01.416579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9ece319d-b5dd-4f45-ab5d-f2daa26dfd5f tempest-VolumesSnapshotTestJSON-1107795008 tempest-VolumesSnapshotTestJSON-1107795008-project] 10.222.0.27 "GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1" status: 404 len: 111 microversion: 2.1 time: 0.031376 Oct 13 04:31:01.416579 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 501/1065] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:31:01 2022] GET /compute/v2.1/servers/bf80e953-e21c-4f85-a8fe-e0377896f0a1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:32:46.724859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:32:46.725534 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:32:46.726392 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 13 04:32:46.726535 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Thu Oct 13 04:32:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 (ip 10.222.0.27) !!! Oct 13 04:32:46.731751 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:46.732241 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:46.741985 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:46.742497 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:46.742932 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:46.743628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:46.744083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:46.744507 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:46.771042 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Going to try to terminate instance {{(pid=118673) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:32:46.842641 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-2f7ff9e1-5c38-4d05-8b7b-c2c04f653ac8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "DELETE /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 204 len: 0 microversion: 2.1 time: 0.119404 Oct 13 04:32:46.843229 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 565/1066] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:32:46 2022] DELETE /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:32:46.850144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=8 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:32:46.850904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:32:46.852904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 13 04:32:46.853067 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Thu Oct 13 04:32:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 (ip 10.222.0.27) !!! Oct 13 04:32:46.858663 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:46.859280 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:46.868771 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:46.869260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:46.869698 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:46.870416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:46.870912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:46.871339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:47.006378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:47.014056 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9838f71a-cef9-41f9-a856-7dbdc7c07b8c tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1573 microversion: 2.1 time: 0.165896 Oct 13 04:32:47.014657 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 502/1067] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:46 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1573 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:48.038164 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:48.038880 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:48.045229 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:48.045703 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:48.046130 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:48.046914 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:48.047359 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:48.047776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:48.185000 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:48.193495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e16c1e2a-15ff-4e86-82aa-6e387196c5fe tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1573 microversion: 2.1 time: 0.159927 Oct 13 04:32:48.194065 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 566/1068] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:48 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1573 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:49.216952 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:49.217661 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:49.220144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR oslo.messaging._drivers.impl_rabbit [-] [d2f0bf5a-f18f-4a06-b810-18455412689a] AMQP server on 10.222.0.27:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 13 04:32:49.225638 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:49.226196 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:49.226760 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:49.227533 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:49.228060 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:49.228486 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:49.312339 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:49.320297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-b5a39ce4-1c8d-473c-8ea4-681a21a14859 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1507 microversion: 2.1 time: 0.107270 Oct 13 04:32:49.320867 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 503/1069] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:49 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1507 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:49.636464 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-eaa14896-2438-4b0d-843d-8e766ff42ad0 req-36ca0d79-ab9a-434a-9a84-c0fa750f1021 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4106e63-3d42-4f62-8eb2-6b97a94b3c71", "name": "network-vif-deleted", "tag": "909e2718-9785-4e2c-ae5c-75201a9f4fc5"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:32:49.647035 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-eaa14896-2438-4b0d-843d-8e766ff42ad0 req-36ca0d79-ab9a-434a-9a84-c0fa750f1021 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:49.647523 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-eaa14896-2438-4b0d-843d-8e766ff42ad0 req-36ca0d79-ab9a-434a-9a84-c0fa750f1021 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:49.647957 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-eaa14896-2438-4b0d-843d-8e766ff42ad0 req-36ca0d79-ab9a-434a-9a84-c0fa750f1021 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:49.660213 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-eaa14896-2438-4b0d-843d-8e766ff42ad0 req-36ca0d79-ab9a-434a-9a84-c0fa750f1021 service nova] Creating event network-vif-deleted:909e2718-9785-4e2c-ae5c-75201a9f4fc5 for instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 on hv-cinder-86099 Oct 13 04:32:49.666420 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-eaa14896-2438-4b0d-843d-8e766ff42ad0 req-36ca0d79-ab9a-434a-9a84-c0fa750f1021 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.174418 Oct 13 04:32:49.667083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 567/1070] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:32:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:32:50.344018 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:50.345326 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:50.353210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:50.353727 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:50.354162 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:50.354912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:50.355370 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:50.356312 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:50.379511 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO oslo.messaging._drivers.impl_rabbit [-] [d2f0bf5a-f18f-4a06-b810-18455412689a] Reconnected to AMQP server on 10.222.0.27:5672 via [amqp] client with port 42376. Oct 13 04:32:50.396059 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:50.396546 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:50.397165 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:50.397792 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.objects.instance [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lazy-loading 'fault' on Instance uuid c4106e63-3d42-4f62-8eb2-6b97a94b3c71 {{(pid=118674) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 13 04:32:50.442318 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:50.449481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c774fff9-140f-405c-a2cb-2b317c81158b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 200 len: 1182 microversion: 2.1 time: 0.109568 Oct 13 04:32:50.450264 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 504/1071] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:50 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 1182 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:51.478135 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:51.478859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:51.485609 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:51.486085 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:51.487396 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:51.488088 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:51.488632 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:51.489191 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:51.503031 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] HTTP exception thrown: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Request method failure captured: Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 HTTP/1.1 Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Q_ICSHk-Z8bXMnzbobXu0Te5KSiKL7PZcTsUHVUplf-WCP2gyKwxHd0dORNKfw8bw5Duc1On24-h1y0rkkHe-cGp2IIPXhJr5mOH07GSHBUDtSEwnEF-mOnuVUUh0rATM4l4oYusb2nBduoZVBaEA9bHNkw2j2djTOoYUaz6HQvVw Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:32:51.504692 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_eb360c69b8c240408cccc0cfbbb709f1", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 4cd203fe04644ace8be512ea2c455774 Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:32:51.506449 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'c4106e63-3d42-4f62-8eb2-6b97a94b3c71'} Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:32:51.507713 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:32:51.509280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:32:51.510732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Returning 404 to user: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:32:51.510732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ebafae7a-4b43-48cc-a83e-90735848a64d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 404 len: 111 microversion: 2.1 time: 0.032146 Oct 13 04:32:51.510732 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 568/1072] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:51 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:32:51.603525 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1686881633", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-22921279", "security_groups": [{"name": "tempest-secgroup-smoke-54814354"}], "block_device_mapping_v2": [{"uuid": "eaecac9c-8d80-4db6-a5aa-0ec9255a3a1e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "2d3f0d2e-0640-4034-90cf-e88b8e787a37"}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:32:51.718513 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] validate_networks() for [('2d3f0d2e-0640-4034-90cf-e88b8e787a37', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:32:52.101102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:32:52.101522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:32:52.101990 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:32:52.102420 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:32:52.102850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:32:52.111790 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:52.112270 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:52.112697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:52.113408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:52.113853 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:52.114295 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:52.130408 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:32:52.134394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for user 4cd203fe04644ace8be512ea2c455774 and project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:32:52.190432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:32:52.192562 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='eaecac9c-8d80-4db6-a5aa-0ec9255a3a1e',volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:32:52.324714 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-7bbbdcfd-1b82-4f72-9cfe-1210ea978a0b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.724970 Oct 13 04:32:52.325374 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 505/1073] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:32:51 2022] POST /compute/v2.1/servers => generated 404 bytes in 726 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:32:52.337409 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-51a9a16a-f9a6-4d3d-ad4c-0fd6e0509d64 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:52.338105 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-51a9a16a-f9a6-4d3d-ad4c-0fd6e0509d64 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:52.392336 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-51a9a16a-f9a6-4d3d-ad4c-0fd6e0509d64 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:52.399964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-51a9a16a-f9a6-4d3d-ad4c-0fd6e0509d64 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1093 microversion: 2.1 time: 0.066433 Oct 13 04:32:52.400629 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 569/1074] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:52 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1093 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:53.423308 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:53.424692 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:53.431744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:53.432437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:53.433028 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:53.433936 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:53.434452 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:53.435047 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:53.509779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:53.518030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-8475ca88-77ce-43f7-a1d8-34b3b019e837 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1199 microversion: 2.1 time: 0.099059 Oct 13 04:32:53.518875 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 506/1075] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:53 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1199 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:54.542109 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:54.542867 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:54.550581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:54.551158 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:54.551665 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:54.552431 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:54.552890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:54.553316 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:54.682858 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:54.691102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-b4179c9a-d4e2-4b1d-bc73-7dfaea4fbcab tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.152840 Oct 13 04:32:54.691663 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 570/1076] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:54 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:54.910474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-00f4a2cc-9a0b-4a5b-859e-3ef607e52453 req-72a80fd4-7a00-4be5-b152-f549b67a8fdb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfc95c6a-358a-4289-a562-7291683ed41d", "tag": "6c9b0e8a-71a2-42e1-a251-7148b4bb52dc"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:32:54.920983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-00f4a2cc-9a0b-4a5b-859e-3ef607e52453 req-72a80fd4-7a00-4be5-b152-f549b67a8fdb service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:54.920983 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-00f4a2cc-9a0b-4a5b-859e-3ef607e52453 req-72a80fd4-7a00-4be5-b152-f549b67a8fdb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:54.921384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-00f4a2cc-9a0b-4a5b-859e-3ef607e52453 req-72a80fd4-7a00-4be5-b152-f549b67a8fdb service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:54.933575 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-00f4a2cc-9a0b-4a5b-859e-3ef607e52453 req-72a80fd4-7a00-4be5-b152-f549b67a8fdb service nova] Creating event network-changed:6c9b0e8a-71a2-42e1-a251-7148b4bb52dc for instance bfc95c6a-358a-4289-a562-7291683ed41d on hv-cinder-86099 Oct 13 04:32:54.938503 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-00f4a2cc-9a0b-4a5b-859e-3ef607e52453 req-72a80fd4-7a00-4be5-b152-f549b67a8fdb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031910 Oct 13 04:32:54.939887 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 507/1077] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:32:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:32:55.714765 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:55.715324 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:55.721805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:55.722320 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:55.722754 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:55.723591 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:55.724043 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:55.724467 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:55.860289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:55.868752 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9d525ddb-4eae-439a-addc-0bc6ea79d53b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.157925 Oct 13 04:32:55.869327 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 571/1078] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:55 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:56.893808 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:56.895152 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:56.901323 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:56.901798 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:56.902248 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:56.902930 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:56.903380 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:56.903799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:57.033601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:57.041484 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-12d01fcf-d95e-4093-b03e-1000c9a1d694 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.152967 Oct 13 04:32:57.042057 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 508/1079] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:56 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:58.064001 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:58.064607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:58.071103 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:58.071581 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:58.072015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:58.072705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:58.073153 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:58.073575 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:58.202270 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:58.210688 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-c45fe8d5-0bd3-4917-827b-34553e8a8cdd tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.150597 Oct 13 04:32:58.211258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 572/1080] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:58 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:32:59.235061 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:32:59.235599 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:32:59.242011 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:59.242522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:59.242968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:59.243652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:32:59.244104 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:32:59.244527 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:32:59.378810 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:32:59.386439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-34cf4630-03ce-4eb3-8137-42042b7cf2b9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.156155 Oct 13 04:32:59.387006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 509/1081] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:32:59 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:00.409983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:00.410623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:00.417760 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:00.417903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:00.418366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:00.419067 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:00.419515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:00.419934 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:00.765504 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:00.773686 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d045361e-7462-42c8-ba2c-99be4efc95e1 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.368549 Oct 13 04:33:00.774258 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 573/1082] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:00 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:01.797836 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:01.798469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:01.805309 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:01.805791 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:01.806244 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:01.806934 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:01.807393 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:01.807824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:01.937207 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:01.945474 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c1152414-b03f-4787-afc3-7401c98cf2d6 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.151566 Oct 13 04:33:01.946045 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 510/1083] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:01 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:02.968758 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:02.969285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:02.971244 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:33:02.977073 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:02.977672 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:02.978102 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:02.978808 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:02.979256 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:02.979673 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:03.105845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:03.114734 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-6f134d3b-86be-4d86-a2d6-0e49bfafcafb tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.150113 Oct 13 04:33:03.115306 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 574/1084] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:02 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:04.137066 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:04.137683 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:04.139677 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:33:04.140331 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:33:04.147460 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:04.147943 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:04.148378 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:04.149071 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:04.149534 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:04.150000 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:04.283289 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:04.291236 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-6fe99eb9-3b7e-49ce-8726-d0e709afc3f5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.158106 Oct 13 04:33:04.291799 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 511/1085] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:04 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:05.314147 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:05.314787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:05.321322 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:05.321805 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:05.322265 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:05.322960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:05.323418 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:05.323845 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:05.455459 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:05.464429 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-59536e06-356e-49c2-a806-67a11db179d4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.154220 Oct 13 04:33:05.465009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 575/1086] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:05 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:06.489489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:06.490165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:06.499774 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:06.500257 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:06.500901 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:06.501646 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:06.502115 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:06.502582 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:06.556915 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-0a07b22f-09d3-4aeb-9d39-e5e0ec7635f1 req-a448e503-6a67-471a-a1ab-738334194279 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfc95c6a-358a-4289-a562-7291683ed41d", "name": "network-vif-plugged", "status": "completed", "tag": "6c9b0e8a-71a2-42e1-a251-7148b4bb52dc"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:33:06.567372 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-0a07b22f-09d3-4aeb-9d39-e5e0ec7635f1 req-a448e503-6a67-471a-a1ab-738334194279 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:06.567860 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-0a07b22f-09d3-4aeb-9d39-e5e0ec7635f1 req-a448e503-6a67-471a-a1ab-738334194279 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:06.568293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-0a07b22f-09d3-4aeb-9d39-e5e0ec7635f1 req-a448e503-6a67-471a-a1ab-738334194279 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:06.579758 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-0a07b22f-09d3-4aeb-9d39-e5e0ec7635f1 req-a448e503-6a67-471a-a1ab-738334194279 service nova] Creating event network-vif-plugged:6c9b0e8a-71a2-42e1-a251-7148b4bb52dc for instance bfc95c6a-358a-4289-a562-7291683ed41d on hv-cinder-86099 Oct 13 04:33:06.585832 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-0a07b22f-09d3-4aeb-9d39-e5e0ec7635f1 req-a448e503-6a67-471a-a1ab-738334194279 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105209 Oct 13 04:33:06.586471 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 512/1087] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:33:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:33:06.642414 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:06.654188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-0c916d50-6259-43d3-ba77-2e6d910ba98d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.169976 Oct 13 04:33:06.655177 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 576/1088] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:06 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:33:07.678648 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:07.679262 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:07.685311 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:07.685794 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:07.686250 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:07.686937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:07.687394 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:07.687815 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:07.824304 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:07.832577 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-acf36d42-885f-421e-be35-e92b7e890c24 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1263 microversion: 2.1 time: 0.158240 Oct 13 04:33:07.833148 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 513/1089] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:07 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1263 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:08.856951 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:08.857611 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:08.864865 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:08.865348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:08.865787 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:08.866515 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:08.866980 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:08.867403 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:08.998289 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:09.006028 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-61773043-0cfe-490e-bdbe-275aa74a9741 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1450 microversion: 2.1 time: 0.153201 Oct 13 04:33:09.006626 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 577/1090] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:08 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1450 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:09.027947 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:33:09.028561 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:33:09.034912 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:09.035935 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:09.036381 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:09.037074 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:09.037531 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:09.037961 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:09.168609 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:33:09.176445 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-09cac898-fd0b-4355-b6e4-8e830358c18a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1450 microversion: 2.1 time: 0.152160 Oct 13 04:33:09.177020 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 514/1091] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:33:09 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1450 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:33:10.993915 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-7ce25293-5cd5-43f7-90c7-4444d6d48ecd req-9e0d0b5d-c4f9-4993-a754-9f524d14813b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfc95c6a-358a-4289-a562-7291683ed41d", "tag": "6c9b0e8a-71a2-42e1-a251-7148b4bb52dc"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:33:11.003596 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7ce25293-5cd5-43f7-90c7-4444d6d48ecd req-9e0d0b5d-c4f9-4993-a754-9f524d14813b service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:33:11.004623 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7ce25293-5cd5-43f7-90c7-4444d6d48ecd req-9e0d0b5d-c4f9-4993-a754-9f524d14813b service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:33:11.005054 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7ce25293-5cd5-43f7-90c7-4444d6d48ecd req-9e0d0b5d-c4f9-4993-a754-9f524d14813b service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:33:11.016776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-7ce25293-5cd5-43f7-90c7-4444d6d48ecd req-9e0d0b5d-c4f9-4993-a754-9f524d14813b service nova] Creating event network-changed:6c9b0e8a-71a2-42e1-a251-7148b4bb52dc for instance bfc95c6a-358a-4289-a562-7291683ed41d on hv-cinder-86099 Oct 13 04:33:11.022749 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-7ce25293-5cd5-43f7-90c7-4444d6d48ecd req-9e0d0b5d-c4f9-4993-a754-9f524d14813b service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032513 Oct 13 04:33:11.023547 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 578/1092] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:33:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 13 04:34:43.922601 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:43.923416 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=68 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:43.928537 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1480103886", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-22921279", "security_groups": [{"name": "tempest-secgroup-smoke-54814354"}], "block_device_mapping_v2": [{"uuid": "d578e0bb-9692-43d1-ac71-4ebfab30fe59", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "2d3f0d2e-0640-4034-90cf-e88b8e787a37"}]}} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:34:44.041851 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.network.neutron [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] validate_networks() for [('2d3f0d2e-0640-4034-90cf-e88b8e787a37', None, None, None, None, None)] {{(pid=118674) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 13 04:34:44.410877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Flavor limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 13 04:34:44.411305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Image limits 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 13 04:34:44.411909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Flavor pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 13 04:34:44.412325 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Image pref 0:0:0 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 13 04:34:44.412768 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.virt.hardware [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118674) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 13 04:34:44.422229 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:44.422724 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:44.423162 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:44.423894 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:44.424348 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:44.424770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:44.441189 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 13 04:34:44.445139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.quota [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Getting quotas for user 4cd203fe04644ace8be512ea2c455774 and project eb360c69b8c240408cccc0cfbbb709f1. Resources: {'cores', 'ram', 'instances'} {{(pid=118674) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 13 04:34:44.501701 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Going to run 1 instances... {{(pid=118674) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 13 04:34:44.503828 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d578e0bb-9692-43d1-ac71-4ebfab30fe59',volume_size=None,volume_type=None)] {{(pid=118674) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 13 04:34:44.630640 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-abad76b3-2637-4db3-a3eb-9461cd613c7b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.709782 Oct 13 04:34:44.631297 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 515/1093] 10.222.0.27 () {60 vars in 1147 bytes} [Thu Oct 13 04:34:43 2022] POST /compute/v2.1/servers => generated 404 bytes in 711 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 13 04:34:44.640927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:44.641897 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:44.642790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 13 04:34:44.643126 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Thu Oct 13 04:34:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c (ip 10.222.0.27) !!! Oct 13 04:34:44.651140 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-01d4d1cd-10da-4cb0-9a3b-871f2df2a2b7 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:44.651245 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-01d4d1cd-10da-4cb0-9a3b-871f2df2a2b7 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:44.711489 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-01d4d1cd-10da-4cb0-9a3b-871f2df2a2b7 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:44.721620 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-01d4d1cd-10da-4cb0-9a3b-871f2df2a2b7 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1093 microversion: 2.1 time: 0.082855 Oct 13 04:34:44.722179 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 579/1094] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:44 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1093 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:45.745481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:45.746327 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:45.752614 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:45.753103 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:45.753537 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:45.754251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:45.754713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:45.755139 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:45.818945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:45.828176 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-9ba4f517-a59e-41e1-bf23-1202585712e3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1199 microversion: 2.1 time: 0.087368 Oct 13 04:34:45.838289 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 516/1095] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:45 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1199 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:46.852081 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:46.852699 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:46.859893 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:46.860377 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:46.860811 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:46.861507 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:46.861958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:46.862484 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:46.992044 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:47.000958 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-4ae5d9da-4b78-49a4-9306-7587399582a0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.152749 Oct 13 04:34:47.001776 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 580/1096] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:46 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:47.478945 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [req-1f9c00dc-64df-48f2-9079-12dfde068764 req-f672dfc3-73cc-4be1-948a-b0b243bc954a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6849b5bd-7176-47a1-8391-1fb384436d6c", "tag": "a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0"}]} {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:34:47.491098 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1f9c00dc-64df-48f2-9079-12dfde068764 req-f672dfc3-73cc-4be1-948a-b0b243bc954a service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:47.491580 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1f9c00dc-64df-48f2-9079-12dfde068764 req-f672dfc3-73cc-4be1-948a-b0b243bc954a service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:47.492163 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [req-1f9c00dc-64df-48f2-9079-12dfde068764 req-f672dfc3-73cc-4be1-948a-b0b243bc954a service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:47.504439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.compute.server_external_events [req-1f9c00dc-64df-48f2-9079-12dfde068764 req-f672dfc3-73cc-4be1-948a-b0b243bc954a service nova] Creating event network-changed:a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0 for instance 6849b5bd-7176-47a1-8391-1fb384436d6c on hv-cinder-86099 Oct 13 04:34:47.509390 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [req-1f9c00dc-64df-48f2-9079-12dfde068764 req-f672dfc3-73cc-4be1-948a-b0b243bc954a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034193 Oct 13 04:34:47.510260 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 517/1097] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:34:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:34:48.025005 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:48.025628 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:48.032003 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:48.032478 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:48.032904 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:48.033585 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:48.034029 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:48.034474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:48.165143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:48.173366 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-e82b6e1d-a84b-4ccf-a772-37a0fd832f8a tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.152277 Oct 13 04:34:48.173931 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 581/1098] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:48 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:49.197512 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:49.198251 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:49.204522 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:49.205005 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:49.205439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:49.206125 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:49.206602 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:49.207023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:49.338144 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:49.346427 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-c006ea05-dcdd-4011-baaa-4ac7d8503296 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.153599 Oct 13 04:34:49.347009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 518/1099] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:49 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:50.369335 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:50.369949 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:50.377009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:50.377493 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:50.377929 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:50.378660 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:50.379113 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:50.379540 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:50.507960 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:50.515584 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1fcdd547-693e-480f-bd7e-72ed00099146 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.150396 Oct 13 04:34:50.516155 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 582/1100] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:50 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:51.538373 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:51.539010 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:51.545030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:51.545510 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:51.545941 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:51.546650 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:51.547110 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:51.547530 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:51.682713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:51.691034 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-baaaf34e-a607-4f47-ad9c-f7abacf4d6cf tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.156726 Oct 13 04:34:51.691606 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 519/1101] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:51 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:52.713725 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:52.714306 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:52.720981 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:52.721495 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:52.721927 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:52.722644 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:52.723098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:52.723516 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:52.850831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:52.859348 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-32ce0441-87e8-41f0-aa69-379ec080d5a3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.149613 Oct 13 04:34:52.860261 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 583/1102] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:52 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:53.881986 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:53.882606 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:53.889854 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:53.890359 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:53.890794 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:53.891481 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:53.891929 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:53.892347 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:54.021593 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:54.029463 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-810d0fc8-c2d1-4693-8161-945a755ef3c0 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.151299 Oct 13 04:34:54.030030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 520/1103] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:53 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:55.052285 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:55.052822 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:55.055143 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell0:SELECT=2 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:55.061303 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:55.061777 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:55.062238 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:55.062909 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:55.063354 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:55.063775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:55.195279 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:55.203780 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-f3984ee7-dba4-4d3a-a36c-ef289e368ebc tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.155406 Oct 13 04:34:55.204389 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 584/1104] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:55 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:56.226652 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:56.227281 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:56.229210 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=2 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:56.230402 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell0:SELECT=6 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:34:56.236620 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:56.237102 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:56.237538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:56.238252 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:56.238713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:56.239137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:56.374702 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:56.382375 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-985f4c2a-7b68-430e-a155-949e7c884c6b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.159564 Oct 13 04:34:56.382944 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 521/1105] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:56 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:57.405294 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:57.405906 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:57.412492 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:57.412977 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:57.413412 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:57.414098 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:57.414782 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:57.415201 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:57.548855 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:57.556441 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-5914890e-70dc-4999-81ab-947d0e2ec050 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.155405 Oct 13 04:34:57.557010 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 585/1106] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:57 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:58.629654 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:58.630305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:58.636678 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:58.637333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:58.637770 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:58.638574 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:58.639030 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:58.639454 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:58.769631 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:58.778658 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-396f6fce-9252-408a-adb8-8d91d3ec0aa5 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.201814 Oct 13 04:34:58.779446 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 522/1107] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:58 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:34:58.908427 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-6cc17628-1eac-47a6-9ad0-23581e3a99d1 req-4e5b8455-141f-4b16-a2b6-91ba5360d994 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6849b5bd-7176-47a1-8391-1fb384436d6c", "name": "network-vif-plugged", "status": "completed", "tag": "a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:34:58.919346 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-6cc17628-1eac-47a6-9ad0-23581e3a99d1 req-4e5b8455-141f-4b16-a2b6-91ba5360d994 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:58.919831 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-6cc17628-1eac-47a6-9ad0-23581e3a99d1 req-4e5b8455-141f-4b16-a2b6-91ba5360d994 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:58.920265 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-6cc17628-1eac-47a6-9ad0-23581e3a99d1 req-4e5b8455-141f-4b16-a2b6-91ba5360d994 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:58.932280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-6cc17628-1eac-47a6-9ad0-23581e3a99d1 req-4e5b8455-141f-4b16-a2b6-91ba5360d994 service nova] Creating event network-vif-plugged:a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0 for instance 6849b5bd-7176-47a1-8391-1fb384436d6c on hv-cinder-86099 Oct 13 04:34:58.938448 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-6cc17628-1eac-47a6-9ad0-23581e3a99d1 req-4e5b8455-141f-4b16-a2b6-91ba5360d994 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034262 Oct 13 04:34:58.939325 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 586/1108] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:34:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:34:59.801998 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:34:59.802651 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:34:59.809790 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:59.810298 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:59.810740 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:59.811437 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:34:59.811895 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:34:59.812322 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:34:59.940840 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:34:59.948469 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-53eae4a4-a417-442b-be75-ddc25a4c70c9 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1263 microversion: 2.1 time: 0.150303 Oct 13 04:34:59.949044 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 523/1109] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:34:59 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1263 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:35:00.971905 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:35:00.972526 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:35:00.979193 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:35:00.979707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:35:00.980178 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:35:00.980924 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:35:00.981376 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:35:00.981801 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:35:01.113654 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:35:01.121685 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-ee457a64-ed70-4710-9c94-57bc93176eca tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1450 microversion: 2.1 time: 0.154072 Oct 13 04:35:01.122280 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 587/1110] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:35:00 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1450 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:35:01.144070 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:35:01.144671 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:35:01.151024 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:35:01.152023 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:35:01.152716 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:35:01.153451 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:35:01.153904 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:35:01.154352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:35:01.284254 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:35:01.293288 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-22388e1a-7355-4306-a583-0d67018a472d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1450 microversion: 2.1 time: 0.152277 Oct 13 04:35:01.294006 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 524/1111] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:35:01 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1450 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:35:02.852610 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-90e1ddea-d29f-4a16-a3f1-0a583d814ab0 req-0d76d89a-6cdd-4acb-b5fb-75ea956b4d62 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6849b5bd-7176-47a1-8391-1fb384436d6c", "tag": "a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:35:02.863009 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-90e1ddea-d29f-4a16-a3f1-0a583d814ab0 req-0d76d89a-6cdd-4acb-b5fb-75ea956b4d62 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:35:02.863491 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-90e1ddea-d29f-4a16-a3f1-0a583d814ab0 req-0d76d89a-6cdd-4acb-b5fb-75ea956b4d62 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:35:02.863918 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-90e1ddea-d29f-4a16-a3f1-0a583d814ab0 req-0d76d89a-6cdd-4acb-b5fb-75ea956b4d62 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:35:02.875312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-90e1ddea-d29f-4a16-a3f1-0a583d814ab0 req-0d76d89a-6cdd-4acb-b5fb-75ea956b4d62 service nova] Creating event network-changed:a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0 for instance 6849b5bd-7176-47a1-8391-1fb384436d6c on hv-cinder-86099 Oct 13 04:35:02.880308 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-90e1ddea-d29f-4a16-a3f1-0a583d814ab0 req-0d76d89a-6cdd-4acb-b5fb-75ea956b4d62 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031588 Oct 13 04:35:02.881218 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 588/1112] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:35:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:36:42.424274 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_api:SELECT=22 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:36:42.424964 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG dbcounter [-] [118673] Writing DB stats nova_cell1:SELECT=46 {{(pid=118673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:36:42.426177 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 13 04:36:42.426607 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Thu Oct 13 04:36:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.27) !!! Oct 13 04:36:42.427282 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:36:42.429933 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG dbcounter [-] [118674] Writing DB stats nova_cell1:SELECT=52 {{(pid=118674) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 04:36:42.433024 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-70a4b8c5-806d-4693-9e7f-2f50a767a705 req-8cc2f4fd-162e-448f-b8fb-49605eab0b5d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6849b5bd-7176-47a1-8391-1fb384436d6c", "tag": "a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:36:42.433482 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 13 04:36:42.433670 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Thu Oct 13 04:36:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c (ip 10.222.0.27) !!! Oct 13 04:36:42.442140 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:42.445703 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-70a4b8c5-806d-4693-9e7f-2f50a767a705 req-8cc2f4fd-162e-448f-b8fb-49605eab0b5d service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:42.446188 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-70a4b8c5-806d-4693-9e7f-2f50a767a705 req-8cc2f4fd-162e-448f-b8fb-49605eab0b5d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:42.446363 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:42.446649 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-70a4b8c5-806d-4693-9e7f-2f50a767a705 req-8cc2f4fd-162e-448f-b8fb-49605eab0b5d service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:42.458909 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:42.459713 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:42.460137 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-70a4b8c5-806d-4693-9e7f-2f50a767a705 req-8cc2f4fd-162e-448f-b8fb-49605eab0b5d service nova] Creating event network-changed:a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0 for instance 6849b5bd-7176-47a1-8391-1fb384436d6c on hv-cinder-86099 Oct 13 04:36:42.460274 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:42.460858 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:42.461319 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:42.461748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:42.466015 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-70a4b8c5-806d-4693-9e7f-2f50a767a705 req-8cc2f4fd-162e-448f-b8fb-49605eab0b5d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044063 Oct 13 04:36:42.470131 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 589/1113] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:36:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:36:42.492748 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:36:42.578141 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-cf5999f3-f4d7-4d5e-ab2f-d9b66847bca8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "DELETE /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 204 len: 0 microversion: 2.1 time: 0.160728 Oct 13 04:36:42.578968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 525/1114] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:36:42 2022] DELETE /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:36:42.590767 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:42.591477 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:42.597941 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:42.598501 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:42.598920 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:42.599612 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:42.600057 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:42.600474 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:42.793387 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:36:42.801331 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-d41f94a1-414c-47bf-936c-aadd657af7f3 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1574 microversion: 2.1 time: 0.217222 Oct 13 04:36:42.801903 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 590/1115] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:42 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1574 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:36:43.831591 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:43.832285 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:43.839293 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:43.839784 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:43.840216 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:43.840902 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:43.841351 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:43.841779 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:44.097352 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:36:44.106009 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-53626d00-d674-45c7-9830-b51f1f13d55b tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1574 microversion: 2.1 time: 0.278671 Oct 13 04:36:44.106612 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 526/1116] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:43 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1574 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:36:44.750935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-099263b9-fc55-475c-a449-3c6ed4244c11 req-cf514296-3671-412e-bb43-4608c2946342 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6849b5bd-7176-47a1-8391-1fb384436d6c", "name": "network-vif-deleted", "tag": "a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:36:44.762650 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-099263b9-fc55-475c-a449-3c6ed4244c11 req-cf514296-3671-412e-bb43-4608c2946342 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:44.763123 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-099263b9-fc55-475c-a449-3c6ed4244c11 req-cf514296-3671-412e-bb43-4608c2946342 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:44.763546 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-099263b9-fc55-475c-a449-3c6ed4244c11 req-cf514296-3671-412e-bb43-4608c2946342 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:44.775264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-099263b9-fc55-475c-a449-3c6ed4244c11 req-cf514296-3671-412e-bb43-4608c2946342 service nova] Creating event network-vif-deleted:a8ee861a-a1cf-4adb-9e0a-54f39bbbe0e0 for instance 6849b5bd-7176-47a1-8391-1fb384436d6c on hv-cinder-86099 Oct 13 04:36:44.782406 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-099263b9-fc55-475c-a449-3c6ed4244c11 req-cf514296-3671-412e-bb43-4608c2946342 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034983 Oct 13 04:36:44.783219 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 591/1117] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:36:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:36:45.129238 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:45.130501 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:45.137849 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:45.138384 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:45.138824 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:45.139533 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:45.140294 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:45.140409 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:45.207258 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:36:45.215085 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-e5e69d2b-e46f-444e-ac43-e4126067f841 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 200 len: 1224 microversion: 2.1 time: 0.089978 Oct 13 04:36:45.215666 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 527/1118] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:45 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 13 04:36:46.241248 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:46.241935 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: 6849b5bd-7176-47a1-8391-1fb384436d6c] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:46.248512 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:46.248987 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:46.249416 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:46.250097 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:46.250565 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:46.250983 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:46.264236 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] HTTP exception thrown: Instance 6849b5bd-7176-47a1-8391-1fb384436d6c could not be found. Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Request method failure captured: Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c HTTP/1.1 Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Q_ICSHk-Z8bXMnzbobXu0Te5KSiKL7PZcTsUHVUplf-WCP2gyKwxHd0dORNKfw8bw5Duc1On24-h1y0rkkHe-cGp2IIPXhJr5mOH07GSHBUDtSEwnEF-mOnuVUUh0rATM4l4oYusb2nBduoZVBaEA9bHNkw2j2djTOoYUaz6HQvVw Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:36:46.265847 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_eb360c69b8c240408cccc0cfbbb709f1", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 4cd203fe04644ace8be512ea2c455774 Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance 6849b5bd-7176-47a1-8391-1fb384436d6c could not be found. Oct 13 04:36:46.267705 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': '6849b5bd-7176-47a1-8391-1fb384436d6c'} Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6849b5bd-7176-47a1-8391-1fb384436d6c could not be found. Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:36:46.268727 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6849b5bd-7176-47a1-8391-1fb384436d6c could not be found. Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6849b5bd-7176-47a1-8391-1fb384436d6c could not be found. Oct 13 04:36:46.270508 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:46.272004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Returning 404 to user: Instance 6849b5bd-7176-47a1-8391-1fb384436d6c could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:36:46.272004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-49d39e74-0a43-473a-9a77-35b0e1d34e5f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c" status: 404 len: 111 microversion: 2.1 time: 0.030611 Oct 13 04:36:46.272004 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 592/1119] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:46 2022] GET /compute/v2.1/servers/6849b5bd-7176-47a1-8391-1fb384436d6c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:36:49.324159 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:49.324767 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:49.329645 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-7feb0bde-d469-4f6d-b591-e38e5eecbf00 req-432294f8-9b43-4b60-883e-d024a088f05e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfc95c6a-358a-4289-a562-7291683ed41d", "tag": "6c9b0e8a-71a2-42e1-a251-7148b4bb52dc"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:36:49.339838 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7feb0bde-d469-4f6d-b591-e38e5eecbf00 req-432294f8-9b43-4b60-883e-d024a088f05e service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:49.340312 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7feb0bde-d469-4f6d-b591-e38e5eecbf00 req-432294f8-9b43-4b60-883e-d024a088f05e service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:49.340738 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-7feb0bde-d469-4f6d-b591-e38e5eecbf00 req-432294f8-9b43-4b60-883e-d024a088f05e service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:49.342568 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:49.343153 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:49.343664 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:49.344439 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:49.345137 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:49.345567 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:49.352251 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-7feb0bde-d469-4f6d-b591-e38e5eecbf00 req-432294f8-9b43-4b60-883e-d024a088f05e service nova] Creating event network-changed:6c9b0e8a-71a2-42e1-a251-7148b4bb52dc for instance bfc95c6a-358a-4289-a562-7291683ed41d on hv-cinder-86099 Oct 13 04:36:49.358466 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-7feb0bde-d469-4f6d-b591-e38e5eecbf00 req-432294f8-9b43-4b60-883e-d024a088f05e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032365 Oct 13 04:36:49.359159 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 593/1120] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:36:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:36:49.374560 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Going to try to terminate instance {{(pid=118674) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 13 04:36:49.462744 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-34ebfdad-b380-4f54-bbb7-acf82266342f tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "DELETE /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 204 len: 0 microversion: 2.1 time: 0.142493 Oct 13 04:36:49.463538 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 528/1121] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:36:49 2022] DELETE /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 13 04:36:49.472859 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:49.473828 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:49.480634 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:49.481120 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:49.481555 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:49.482275 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:49.482731 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:49.483155 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:49.621748 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.policy [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118673) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:36:49.629913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-030b68dc-f375-4d03-a92a-b514b35b74c8 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1574 microversion: 2.1 time: 0.161447 Oct 13 04:36:49.630545 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 594/1122] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:49 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1574 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:36:50.656877 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:50.657595 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:50.663937 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:50.664418 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:50.664847 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:50.665528 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:50.665976 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:50.666428 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:50.797850 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:36:50.805626 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-33893520-840e-4dde-a91f-4c84a78f1355 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1574 microversion: 2.1 time: 0.154825 Oct 13 04:36:50.806195 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 529/1123] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:50 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1574 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:36:51.719856 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [req-00020b3c-0cc0-4af6-95f7-6aec276ca615 req-d6471429-70a8-445e-9b4f-4983a29d8cb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfc95c6a-358a-4289-a562-7291683ed41d", "name": "network-vif-deleted", "tag": "6c9b0e8a-71a2-42e1-a251-7148b4bb52dc"}]} {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 13 04:36:51.731529 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-00020b3c-0cc0-4af6-95f7-6aec276ca615 req-d6471429-70a8-445e-9b4f-4983a29d8cb8 service nova] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:51.732002 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-00020b3c-0cc0-4af6-95f7-6aec276ca615 req-d6471429-70a8-445e-9b4f-4983a29d8cb8 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:51.732429 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [req-00020b3c-0cc0-4af6-95f7-6aec276ca615 req-d6471429-70a8-445e-9b4f-4983a29d8cb8 service nova] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:51.744502 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.compute.server_external_events [req-00020b3c-0cc0-4af6-95f7-6aec276ca615 req-d6471429-70a8-445e-9b4f-4983a29d8cb8 service nova] Creating event network-vif-deleted:6c9b0e8a-71a2-42e1-a251-7148b4bb52dc for instance bfc95c6a-358a-4289-a562-7291683ed41d on hv-cinder-86099 Oct 13 04:36:51.750624 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [req-00020b3c-0cc0-4af6-95f7-6aec276ca615 req-d6471429-70a8-445e-9b4f-4983a29d8cb8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034514 Oct 13 04:36:51.751222 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 595/1124] 10.222.0.27 () {64 vars in 1317 bytes} [Thu Oct 13 04:36:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 13 04:36:51.829646 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:51.830432 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:51.836723 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:51.837283 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:51.837697 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:51.838417 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:51.839588 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:51.840305 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:51.904584 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.policy [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4cd203fe04644ace8be512ea2c455774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eb360c69b8c240408cccc0cfbbb709f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118674) authorize /opt/stack/nova/nova/policy.py:199}} Oct 13 04:36:51.912400 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-35891e62-ef5d-4497-9ba5-86fdafbe2809 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 200 len: 1224 microversion: 2.1 time: 0.086625 Oct 13 04:36:51.912978 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 530/1125] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:51 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 13 04:36:52.937119 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:52.937726 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: bfc95c6a-358a-4289-a562-7291683ed41d] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:52.943790 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:52.944264 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:52.944700 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:52.945375 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:52.945820 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:52.946259 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:52.959198 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] HTTP exception thrown: Instance bfc95c6a-358a-4289-a562-7291683ed41d could not be found. Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Request method failure captured: Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d HTTP/1.1 Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Q_ICSHk-Z8bXMnzbobXu0Te5KSiKL7PZcTsUHVUplf-WCP2gyKwxHd0dORNKfw8bw5Duc1On24-h1y0rkkHe-cGp2IIPXhJr5mOH07GSHBUDtSEwnEF-mOnuVUUh0rATM4l4oYusb2nBduoZVBaEA9bHNkw2j2djTOoYUaz6HQvVw Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:36:52.960783 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_eb360c69b8c240408cccc0cfbbb709f1", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 4cd203fe04644ace8be512ea2c455774 Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance bfc95c6a-358a-4289-a562-7291683ed41d could not be found. Oct 13 04:36:52.962536 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'bfc95c6a-358a-4289-a562-7291683ed41d'} Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bfc95c6a-358a-4289-a562-7291683ed41d could not be found. Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:36:52.963640 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bfc95c6a-358a-4289-a562-7291683ed41d could not be found. Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bfc95c6a-358a-4289-a562-7291683ed41d could not be found. Oct 13 04:36:52.966083 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:52.967707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Returning 404 to user: Instance bfc95c6a-358a-4289-a562-7291683ed41d could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:36:52.967707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-1c793a6e-0429-4ca2-b68a-878498fef48d tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d" status: 404 len: 111 microversion: 2.1 time: 0.029932 Oct 13 04:36:52.967707 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 596/1126] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:52 2022] GET /compute/v2.1/servers/bfc95c6a-358a-4289-a562-7291683ed41d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:36:53.398846 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:53.399488 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.compute.api [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118674) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:53.406719 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:53.407203 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:53.407637 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:53.408333 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:53.408844 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:53.409545 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG oslo_concurrency.lockutils [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118674) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:53.422843 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.wsgi [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] HTTP exception thrown: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Request method failure captured: Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: request: DELETE /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 HTTP/1.1 Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept: application/json Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Accept-Encoding: identity Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Connection: close Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Length: 0 Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Content-Type: application/json Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Host: 10.222.0.27 Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: Openstack-System-Scope: None Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: User-Agent: python-urllib3/1.26.12 Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Auth-Token: gAAAAABjR5Q_ICSHk-Z8bXMnzbobXu0Te5KSiKL7PZcTsUHVUplf-WCP2gyKwxHd0dORNKfw8bw5Duc1On24-h1y0rkkHe-cGp2IIPXhJr5mOH07GSHBUDtSEwnEF-mOnuVUUh0rATM4l4oYusb2nBduoZVBaEA9bHNkw2j2djTOoYUaz6HQvVw Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Id: None Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Domain-Name: None Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Identity-Status: Confirmed Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Is-Admin-Project: True Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Id: default Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Domain-Name: Default Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Project-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Role: member,reader Oct 13 04:36:53.424505 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Roles: member,reader Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_eb360c69b8c240408cccc0cfbbb709f1", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-Tenant-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Id: default Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Domain-Name: Default Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Id: 4cd203fe04644ace8be512ea2c455774 Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: X-User-Name: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: method: > Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: exception: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.426296 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: action_args: {'id': 'c4106e63-3d42-4f62-8eb2-6b97a94b3c71'} Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:36:53.427278 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.428891 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.430968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Returning 404 to user: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. {{(pid=118674) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:36:53.430968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-189c4ef8-5aea-4bf2-b702-151c1856a8e4 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "DELETE /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 404 len: 111 microversion: 2.1 time: 0.030868 Oct 13 04:36:53.430968 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 531/1127] 10.222.0.27 () {58 vars in 1239 bytes} [Thu Oct 13 04:36:53 2022] DELETE /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:36:53.434730 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '>' {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:53.435337 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.compute.api [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] [instance: c4106e63-3d42-4f62-8eb2-6b97a94b3c71] Fetching instance by UUID {{(pid=118673) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 13 04:36:53.441181 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:53.441658 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:53.442087 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:53.442802 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Acquiring lock "1311a648-eb92-4594-8c2f-3c55fab1c340" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 13 04:36:53.443250 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 13 04:36:53.443675 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG oslo_concurrency.lockutils [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Lock "1311a648-eb92-4594-8c2f-3c55fab1c340" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118673) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 13 04:36:53.457635 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.wsgi [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] HTTP exception thrown: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Request method failure captured: Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: request: GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 HTTP/1.1 Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept: application/json Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Accept-Encoding: identity Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Connection: close Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Length: 0 Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Content-Type: application/json Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Host: 10.222.0.27 Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: Openstack-System-Scope: None Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: User-Agent: python-urllib3/1.26.12 Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Auth-Token: gAAAAABjR5Q_ICSHk-Z8bXMnzbobXu0Te5KSiKL7PZcTsUHVUplf-WCP2gyKwxHd0dORNKfw8bw5Duc1On24-h1y0rkkHe-cGp2IIPXhJr5mOH07GSHBUDtSEwnEF-mOnuVUUh0rATM4l4oYusb2nBduoZVBaEA9bHNkw2j2djTOoYUaz6HQvVw Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Id: None Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Domain-Name: None Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Identity-Status: Confirmed Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Is-Admin-Project: True Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Id: default Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Domain-Name: Default Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Project-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Role: member,reader Oct 13 04:36:53.459165 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Roles: member,reader Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/eb360c69b8c240408cccc0cfbbb709f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_eb360c69b8c240408cccc0cfbbb709f1", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}] Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Id: eb360c69b8c240408cccc0cfbbb709f1 Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-Tenant-Name: tempest-TestVolumeBootPattern-1504002684 Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Id: default Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Domain-Name: Default Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Id: 4cd203fe04644ace8be512ea2c455774 Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: X-User-Name: tempest-TestVolumeBootPattern-1504002684-project Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: method: > Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: exception: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.460913 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: action_args: {'id': 'c4106e63-3d42-4f62-8eb2-6b97a94b3c71'} Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: {{(pid=118673) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 13 04:36:53.461890 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. Oct 13 04:36:53.463775 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: ERROR nova.api.openstack.wsgi Oct 13 04:36:53.466114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: DEBUG nova.api.openstack.wsgi [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Returning 404 to user: Instance c4106e63-3d42-4f62-8eb2-6b97a94b3c71 could not be found. {{(pid=118673) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 13 04:36:53.466114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: INFO nova.api.openstack.requestlog [None req-9a443b58-5afc-432e-aa6c-d719182a6f91 tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71" status: 404 len: 111 microversion: 2.1 time: 0.029836 Oct 13 04:36:53.466114 dv-cinder-860997-1-iscsi devstack@n-api.service[118673]: [pid: 118673|app: 0|req: 597/1128] 10.222.0.27 () {58 vars in 1236 bytes} [Thu Oct 13 04:36:53 2022] GET /compute/v2.1/servers/c4106e63-3d42-4f62-8eb2-6b97a94b3c71 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 13 04:36:54.756796 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: DEBUG nova.api.openstack.wsgi [None req-a15274de-f243-474b-a7d4-ddb2716a2b5e tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] Calling method '.version_select at 0x7f07949d8820>' {{(pid=118674) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 13 04:36:54.780966 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: INFO nova.api.openstack.requestlog [None req-a15274de-f243-474b-a7d4-ddb2716a2b5e tempest-TestVolumeBootPattern-1504002684 tempest-TestVolumeBootPattern-1504002684-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-22921279" status: 202 len: 0 microversion: 2.1 time: 0.027916 Oct 13 04:36:54.781546 dv-cinder-860997-1-iscsi devstack@n-api.service[118674]: [pid: 118674|app: 0|req: 532/1129] 10.222.0.27 () {58 vars in 1257 bytes} [Thu Oct 13 04:36:54 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-22921279 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)